Spaces:
Running
Running
update text
Browse files- introduction.md +2 -5
introduction.md
CHANGED
@@ -131,9 +131,6 @@ The multilingual CLIP (henceforth, mCLIP), is a model introduced by [Nils Reimer
|
|
131 |
[sentence-transformer](https://www.sbert.net/index.html) library. mCLIP is based on a multilingual encoder
|
132 |
that was created through multilingual knowledge distillation (see [Reimers et al., 2020](https://aclanthology.org/2020.emnlp-main.365/)).
|
133 |
|
134 |
-
### Experiments Replication
|
135 |
-
We provide two colab notebooks to replicate both experiments.
|
136 |
-
|
137 |
### Tasks
|
138 |
|
139 |
We selected two different tasks:
|
@@ -152,7 +149,7 @@ Both experiments should be very easy to replicate, we share the two colab notebo
|
|
152 |
|
153 |
This experiment is run against the MSCOCO-IT validation set (that we haven't used in training). Given in input
|
154 |
a caption, we search for the most similar image in the MSCOCO-IT validation set. As evaluation metrics
|
155 |
-
we use the MRR.
|
156 |
|
157 |
| MRR | CLIP-Italian | mCLIP |
|
158 |
| --------------- | ------------ |-------|
|
@@ -167,7 +164,7 @@ on 400million images (and some of them probably were from MSCOCO).
|
|
167 |
### Zero-shot image classification
|
168 |
|
169 |
This experiment replicates the original one run by OpenAI on zero-shot image classification on ImageNet.
|
170 |
-
To do this, we used DeepL to translate the image labels in ImageNet. We evaluate the models computing the accuracy.
|
171 |
|
172 |
|
173 |
| Accuracy | CLIP-Italian | mCLIP |
|
|
|
131 |
[sentence-transformer](https://www.sbert.net/index.html) library. mCLIP is based on a multilingual encoder
|
132 |
that was created through multilingual knowledge distillation (see [Reimers et al., 2020](https://aclanthology.org/2020.emnlp-main.365/)).
|
133 |
|
|
|
|
|
|
|
134 |
### Tasks
|
135 |
|
136 |
We selected two different tasks:
|
|
|
149 |
|
150 |
This experiment is run against the MSCOCO-IT validation set (that we haven't used in training). Given in input
|
151 |
a caption, we search for the most similar image in the MSCOCO-IT validation set. As evaluation metrics
|
152 |
+
we use the MRR@K.
|
153 |
|
154 |
| MRR | CLIP-Italian | mCLIP |
|
155 |
| --------------- | ------------ |-------|
|
|
|
164 |
### Zero-shot image classification
|
165 |
|
166 |
This experiment replicates the original one run by OpenAI on zero-shot image classification on ImageNet.
|
167 |
+
To do this, we used DeepL to translate the image labels in ImageNet. We evaluate the models computing the accuracy at different levels.
|
168 |
|
169 |
|
170 |
| Accuracy | CLIP-Italian | mCLIP |
|