File size: 1,725 Bytes
bcff2b5 61fe531 bcff2b5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
---
tags:
- BERT
- Cebuano
---
## Model Description
As part of the ITANONG project's 10 billion-token Tagalog dataset, we have introduced our initial pre-trained language models for Philippine languages. Our model suite encompasses various BERT-based, GPT-based, and Sentence Transformers tailored for Tagalog,Taglish and Cebuano.
## Training Details
This model was trained using an Nvidia V100-32GB GPU on DOST-ASTI Computing and Archiving Research Environment (COARE) - https://asti.dost.gov.ph/projects/coare/
### Training Data
The training dataset was compiled from both formal and informal sources, consisting of 194,001 instances from formal channels and 1,816,735 from informal sources. More information on pre-processing and training parameters on our paper
## Citation
Paper : iTANONG-DS : A Collection of Benchmark Datasets for Downstream Natural Language Processing Tasks on Select Philippine Language
Bibtex:
```
@inproceedings{visperas-etal-2023-itanong,
title = "i{TANONG}-{DS} : A Collection of Benchmark Datasets for Downstream Natural Language Processing Tasks on Select {P}hilippine Languages",
author = "Visperas, Moses L. and
Borjal, Christalline Joie and
Adoptante, Aunhel John M and
Abacial, Danielle Shine R. and
Decano, Ma. Miciella and
Peramo, Elmer C",
editor = "Abbas, Mourad and
Freihat, Abed Alhakim",
booktitle = "Proceedings of the 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023)",
month = dec,
year = "2023",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.icnlsp-1.34",
pages = "316--323",
}
``` |