add model

README.md

@@ -1,67 +1,47 @@
 ---
-language:
-- ga
-license: apache-2.0
 tags:
-- irish
-- bert
-widget:
-- text: "Ceoltóir [MASK] ab ea Johnny Cash."
+- generated_from_keras_callback
+model-index:
+- name: bert-base-irish-cased-v1
+  results: []
 ---
 
-# gaBERT
+<!-- This model card has been generated automatically according to the information Keras had access to. You should
+probably proofread and complete it, then remove this comment. -->
 
-[gaBERT](https://arxiv.org/abs/2107.12930) is a BERT-base model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper.
+# bert-base-irish-cased-v1
 
-### How to use gaBERT with HuggingFace
+This model was trained from scratch on an unknown dataset.
+It achieves the following results on the evaluation set:
 
-```
-from transformers import AutoModelWithLMHead, AutoTokenizer
-import torch
 
-tokenizer = AutoTokenizer.from_pretrained("DCU-NLP/bert-base-irish-cased-v1")
-model = AutoModelWithLMHead.from_pretrained("DCU-NLP/bert-base-irish-cased-v1")
+## Model description
 
-sequence = f"Ceoltóir {tokenizer.mask_token} ab ea Johnny Cash."
+More information needed
 
-input = tokenizer.encode(sequence, return_tensors="pt")
-mask_token_index = torch.where(input == tokenizer.mask_token_id)[1]
+## Intended uses & limitations
 
-token_logits = model(input)[0]
-mask_token_logits = token_logits[0, mask_token_index, :]
+More information needed
 
-top_5_tokens = torch.topk(mask_token_logits, 5, dim=1).indices[0].tolist()
+## Training and evaluation data
 
-for token in top_5_tokens:
-    print(sequence.replace(tokenizer.mask_token, tokenizer.decode([token])))
-```
+More information needed
 
-### Limitations and bias
-Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.
+## Training procedure
 
-### BibTeX entry and citation info
+### Training hyperparameters
 
-If you use this model in your research, please consider citing our paper:
+The following hyperparameters were used during training:
+- optimizer: None
+- training_precision: float32
 
-```
-@article{DBLP:journals/corr/abs-2107-12930,
-  author    = {James Barry and
-               Joachim Wagner and
-               Lauren Cassidy and
-               Alan Cowap and
-               Teresa Lynn and
-               Abigail Walsh and
-               M{\'{\i}}che{\'{a}}l J. {\'{O}} Meachair and
-               Jennifer Foster},
-  title     = {gaBERT - an Irish Language Model},
-  journal   = {CoRR},
-  volume    = {abs/2107.12930},
-  year      = {2021},
-  url       = {https://arxiv.org/abs/2107.12930},
-  archivePrefix = {arXiv},
-  eprint    = {2107.12930},
-  timestamp = {Fri, 30 Jul 2021 13:03:06 +0200},
-  biburl    = {https://dblp.org/rec/journals/corr/abs-2107-12930.bib},
-  bibsource = {dblp computer science bibliography, https://dblp.org}
-}
-```
+### Training results
+
+
+
+### Framework versions
+
+- Transformers 4.20.1
+- TensorFlow 2.9.1
+- Datasets 2.3.2
+- Tokenizers 0.12.1

config.json

@@ -1,8 +1,10 @@
 {
+  "_name_or_path": "DCU-NLP/bert-base-irish-cased-v1",
   "architectures": [
     "BertForMaskedLM"
   ],
   "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
   "hidden_act": "gelu",
   "hidden_dropout_prob": 0.1,
   "hidden_size": 768,
@@ -14,6 +16,9 @@
   "num_attention_heads": 12,
   "num_hidden_layers": 12,
   "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.20.1",
   "type_vocab_size": 2,
+  "use_cache": true,
   "vocab_size": 30101
 }

tf_model.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5147ad2ba231a4788e05f18e23c3c01f0b8f1d01d600a962ae690afd44d4de8
+size 531099308