julien-c HF staff commited on
Commit
403f435
1 Parent(s): 51cd817

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/mrm8488/electricidad-small-finetuned-squadv1-es/README.md

Files changed (1) hide show
  1. README.md +102 -0
README.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: es
3
+ thumbnail: https://imgur.com/uxAvBfh
4
+ ---
5
+
6
+ # Electricidad small + Spanish SQuAD v1 ⚡❓
7
+
8
+ [Electricidad-small-discriminator](https://huggingface.co/mrm8488/electricidad-small-discriminator) fine-tuned on [Spanish SQUAD v1.1 dataset](https://github.com/ccasimiro88/TranslateAlignRetrieve/tree/master/SQuAD-es-v1.1) for **Q&A** downstream task.
9
+
10
+ ## Details of the downstream task (Q&A) - Dataset 📚
11
+
12
+ [SQuAD-es-v1.1](https://github.com/ccasimiro88/TranslateAlignRetrieve/tree/master/SQuAD-es-v1.1)
13
+
14
+ | Dataset split | # Samples |
15
+ | ------------- | --------- |
16
+ | Train | 130 K |
17
+ | Test | 11 K |
18
+
19
+ ## Model training 🏋️‍
20
+
21
+ The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command:
22
+
23
+ ```bash
24
+ python /content/transformers/examples/question-answering/run_squad.py \
25
+ --model_type electra \
26
+ --model_name_or_path 'mrm8488/electricidad-small-discriminator' \
27
+ --do_eval \
28
+ --do_train \
29
+ --do_lower_case \
30
+ --train_file '/content/dataset/train-v1.1-es.json' \
31
+ --predict_file '/content/dataset/dev-v1.1-es.json' \
32
+ --per_gpu_train_batch_size 16 \
33
+ --learning_rate 3e-5 \
34
+ --num_train_epochs 10 \
35
+ --max_seq_length 384 \
36
+ --doc_stride 128 \
37
+ --output_dir '/content/electricidad-small-finetuned-squadv1-es' \
38
+ --overwrite_output_dir \
39
+ --save_steps 1000
40
+ ```
41
+
42
+ ## Test set Results 🧾
43
+
44
+ | Metric | # Value |
45
+ | ------ | --------- |
46
+ | **EM** | **46.82** |
47
+ | **F1** | **64.79** |
48
+
49
+ ```json
50
+ {
51
+ 'exact': 46.82119205298013,
52
+ 'f1': 64.79435260021918,
53
+ 'total': 10570,
54
+ 'HasAns_exact': 46.82119205298013,
55
+ HasAns_f1': 64.79435260021918,
56
+ 'HasAns_total': 10570,
57
+ 'best_exact': 46.82119205298013,
58
+ 'best_exact_thresh': 0.0,
59
+ 'best_f1': 64.79435260021918,
60
+ 'best_f1_thresh': 0.0
61
+ }
62
+ ```
63
+
64
+ ### Model in action 🚀
65
+
66
+ Fast usage with **pipelines**:
67
+
68
+ ```python
69
+ from transformers import pipeline
70
+
71
+ qa_pipeline = pipeline(
72
+ "question-answering",
73
+ model="mrm8488/electricidad-small-finetuned-squadv1-es",
74
+ tokenizer="mrm8488/electricidad-small-finetuned-squadv1-es"
75
+ )
76
+
77
+ context = "Manuel ha creado una versión del modelo Electra small en español que alcanza una puntuación F1 de 65 en el dataset SQUAD-es y sólo pesa 50 MB"
78
+
79
+ q1 = "Cuál es su marcador F1?"
80
+ q2 = "¿Cuál es el tamaño del modelo?"
81
+ q3 = "¿Quién lo ha creado?"
82
+ q4 = "¿Que es lo que ha hecho Manuel?"
83
+
84
+
85
+ questions = [q1, q2, q3, q4]
86
+
87
+ for question in questions:
88
+ result = qa_pipeline({
89
+ 'context': context,
90
+ 'question': question})
91
+ print(result)
92
+
93
+ # Output:
94
+ {'score': 0.14836778166355025, 'start': 98, 'end': 100, 'answer': '65'}
95
+ {'score': 0.32219420810758237, 'start': 136, 'end': 140, 'answer': '50 MB'}
96
+ {'score': 0.9672326951118713, 'start': 0, 'end': 6, 'answer': 'Manuel'}
97
+ {'score': 0.23552458113848118, 'start': 10, 'end': 53, 'answer': 'creado una versión del modelo Electra small'}
98
+ ```
99
+
100
+ > Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/)
101
+
102
+ > Made with <span style="color: #e25555;">&hearts;</span> in Spain