Malaysian MaskLM
Collection
Trained on 17B tokens, 81GB of cleaned texts, able to understand standard Malay, local Malay, local Mandarin, Manglish, and local Tamil.
•
7 items
•
Updated
Malaysian Mistral 191M on MLM task using 512 context length
Replicating https://github.com/McGill-NLP/llm2vec using https://huggingface.co/mesolitica/malaysian-mistral-191M-4096, done by https://github.com/aisyahrzk https://twitter.com/aisyahhhrzk
Source code at https://github.com/mesolitica/malaya/tree/master/session/llm2vec
WandB, https://wandb.ai/aisyahrazak/mistral-191M-mlm?nw=nwuseraisyahrazak
- Downloads last month
- 23
Inference API (serverless) does not yet support model repos that contain custom code.