juliensimon
commited on
Commit
•
d322b88
1
Parent(s):
8e2a7b5
Update README.md
Browse files
README.md
CHANGED
@@ -3,6 +3,7 @@ license: mit
|
|
3 |
tags:
|
4 |
- generated_from_trainer
|
5 |
- language-identification
|
|
|
6 |
datasets:
|
7 |
- fleurs
|
8 |
metrics:
|
@@ -23,6 +24,7 @@ model-index:
|
|
23 |
- name: Accuracy
|
24 |
type: accuracy
|
25 |
value: 0.9930337861372344
|
|
|
26 |
---
|
27 |
|
28 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
@@ -30,14 +32,45 @@ should probably proofread and complete it, then remove this comment. -->
|
|
30 |
|
31 |
# xlm-v-base-language-id
|
32 |
|
33 |
-
This model is a fine-tuned version of [facebook/xlm-v-base](https://huggingface.co/facebook/xlm-v-base) on the fleurs dataset.
|
34 |
It achieves the following results on the evaluation set:
|
35 |
- Loss: 0.0241
|
36 |
- Accuracy: 0.9930
|
37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
38 |
## Intended uses & limitations
|
39 |
|
40 |
-
The model can accurately detect 102 languages.
|
41 |
|
42 |
## Training and evaluation data
|
43 |
|
@@ -78,4 +111,4 @@ The following hyperparameters were used during training:
|
|
78 |
- Transformers 4.26.0
|
79 |
- Pytorch 1.13.1
|
80 |
- Datasets 2.8.0
|
81 |
-
- Tokenizers 0.13.2
|
|
|
3 |
tags:
|
4 |
- generated_from_trainer
|
5 |
- language-identification
|
6 |
+
- openvino
|
7 |
datasets:
|
8 |
- fleurs
|
9 |
metrics:
|
|
|
24 |
- name: Accuracy
|
25 |
type: accuracy
|
26 |
value: 0.9930337861372344
|
27 |
+
pipeline_tag: text-classification
|
28 |
---
|
29 |
|
30 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
|
|
32 |
|
33 |
# xlm-v-base-language-id
|
34 |
|
35 |
+
This model is a fine-tuned version of [facebook/xlm-v-base](https://huggingface.co/facebook/xlm-v-base) on the [google/fleurs](https://huggingface.co/datasets/google/fleurs) dataset.
|
36 |
It achieves the following results on the evaluation set:
|
37 |
- Loss: 0.0241
|
38 |
- Accuracy: 0.9930
|
39 |
|
40 |
+
# Usage
|
41 |
+
|
42 |
+
The simplest way to use the model is with a text classification pipeline:
|
43 |
+
|
44 |
+
```
|
45 |
+
from transformers import pipeline
|
46 |
+
|
47 |
+
model_id = "juliensimon/xlm-v-base-language-id"
|
48 |
+
p = pipeline("text-classification", model=model_id)
|
49 |
+
p("Hello world")
|
50 |
+
# [{'label': 'English', 'score': 0.9802148342132568}]
|
51 |
+
```
|
52 |
+
|
53 |
+
The model is also compatible with [Optimum Intel](https://github.com/huggingface/optimum-intel).
|
54 |
+
For example, you can optimize it with Intel OpenVINO and enjoy a 2x inference speedup (or more).
|
55 |
+
|
56 |
+
```
|
57 |
+
from optimum.intel.openvino import OVModelForSequenceClassification
|
58 |
+
from transformers import (AutoModelForSequenceClassification, AutoTokenizer,
|
59 |
+
pipeline)
|
60 |
+
|
61 |
+
model_id = "juliensimon/xlm-v-base-language-id"
|
62 |
+
ov_model = OVModelForSequenceClassification.from_pretrained(
|
63 |
+
model_id, from_transformers=True
|
64 |
+
)
|
65 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
66 |
+
p = pipeline("text-classification", model=ov_model, tokenizer=tokenizer)
|
67 |
+
p("Hello world")
|
68 |
+
# [{'label': 'English', 'score': 0.9802149534225464}]
|
69 |
+
```
|
70 |
+
|
71 |
## Intended uses & limitations
|
72 |
|
73 |
+
The model can accurately detect 102 languages. You can find the list on the [dataset](https://huggingface.co/datasets/google/fleurs) page.
|
74 |
|
75 |
## Training and evaluation data
|
76 |
|
|
|
111 |
- Transformers 4.26.0
|
112 |
- Pytorch 1.13.1
|
113 |
- Datasets 2.8.0
|
114 |
+
- Tokenizers 0.13.2
|