MaLA-500: Massive Language Adaptation of Large Language Models
akhaliq
Abstract
Large language models have advanced the state of the art in natural language processing. However, their predominant design for English or a limited set of languages creates a substantial gap in their effectiveness for low-resource languages. To bridge this gap, we introduce MaLA-500, a novel large language model designed to cover an extensive range of 534 languages. To train MaLA-500, we employ vocabulary extension and continued pretraining on LLaMA 2 with Glot500-c. Our experiments on SIB-200 show that MaLA-500 achieves state-of-the-art in-context learning results. We release MaLA-500 at https://huggingface.co/MaLA-LM
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- LLaMA Beyond English: An Empirical Study on Language Capability Transfer (2024)
- PersianMind: A Cross-Lingual Persian-English Large Language Model (2024)
- A Simple Framework to Accelerate Multilingual Language Model for Monolingual Text Generation (2024)
- Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed? (2023)
- LangBridge: Multilingual Reasoning Without Multilingual Supervision (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
Models citing this paper 2
Datasets citing this paper 0
No dataset linking this paper