wangchanberta-base-att-spm-uncased / tokenizer_config.json

Commit History

Merge branch 'main' of https://huggingface.co/airesearch/wangchanberta-base-att-spm-uncased into main
9d1eba6

lalital commited on

Add additional_special_tokens
a85d3a4

lalital commited on

Add additional_special_tokens
3815314

lalital commited on

Remove trailing comma
1049ef5

lalital commited on

Remove "additional_special_tokens" from tokenizer_config.json
77632ea

lalital commited on

Remove pretrained name
926f33a

lalital commited on

Change space token from <th_roberta_space_token> to <_>
04ff256

lalital commited on

Add pretrained_model_name_or_path
fd76d90

lalital commited on

Add vocab file and tokenizer_config.json
247d65f

lalital commited on