vinid commited on
Commit
3a49440
1 Parent(s): f5ab7cc

update text

Browse files
Files changed (1) hide show
  1. introduction.md +2 -5
introduction.md CHANGED
@@ -131,9 +131,6 @@ The multilingual CLIP (henceforth, mCLIP), is a model introduced by [Nils Reimer
131
  [sentence-transformer](https://www.sbert.net/index.html) library. mCLIP is based on a multilingual encoder
132
  that was created through multilingual knowledge distillation (see [Reimers et al., 2020](https://aclanthology.org/2020.emnlp-main.365/)).
133
 
134
- ### Experiments Replication
135
- We provide two colab notebooks to replicate both experiments.
136
-
137
  ### Tasks
138
 
139
  We selected two different tasks:
@@ -152,7 +149,7 @@ Both experiments should be very easy to replicate, we share the two colab notebo
152
 
153
  This experiment is run against the MSCOCO-IT validation set (that we haven't used in training). Given in input
154
  a caption, we search for the most similar image in the MSCOCO-IT validation set. As evaluation metrics
155
- we use the MRR.
156
 
157
  | MRR | CLIP-Italian | mCLIP |
158
  | --------------- | ------------ |-------|
@@ -167,7 +164,7 @@ on 400million images (and some of them probably were from MSCOCO).
167
  ### Zero-shot image classification
168
 
169
  This experiment replicates the original one run by OpenAI on zero-shot image classification on ImageNet.
170
- To do this, we used DeepL to translate the image labels in ImageNet. We evaluate the models computing the accuracy.
171
 
172
 
173
  | Accuracy | CLIP-Italian | mCLIP |
 
131
  [sentence-transformer](https://www.sbert.net/index.html) library. mCLIP is based on a multilingual encoder
132
  that was created through multilingual knowledge distillation (see [Reimers et al., 2020](https://aclanthology.org/2020.emnlp-main.365/)).
133
 
 
 
 
134
  ### Tasks
135
 
136
  We selected two different tasks:
 
149
 
150
  This experiment is run against the MSCOCO-IT validation set (that we haven't used in training). Given in input
151
  a caption, we search for the most similar image in the MSCOCO-IT validation set. As evaluation metrics
152
+ we use the MRR@K.
153
 
154
  | MRR | CLIP-Italian | mCLIP |
155
  | --------------- | ------------ |-------|
 
164
  ### Zero-shot image classification
165
 
166
  This experiment replicates the original one run by OpenAI on zero-shot image classification on ImageNet.
167
+ To do this, we used DeepL to translate the image labels in ImageNet. We evaluate the models computing the accuracy at different levels.
168
 
169
 
170
  | Accuracy | CLIP-Italian | mCLIP |