woody72
/

multilingual-e5-base

Sentence Similarity

sentence-transformers

Sentence Transformers

text-embeddings-inference

Inference Endpoints

Model card Files Files and versions Community

root commited on Nov 5, 2023

Commit

7751418

•

1 Parent(s): c88fe71

added truncation

Files changed (1) hide show

tokenizer_config.json +2 -1

tokenizer_config.json CHANGED Viewed

@@ -15,5 +15,6 @@
   "pad_token": "<pad>",
   "sep_token": "</s>",
   "tokenizer_class": "XLMRobertaTokenizer",
-  "unk_token": "<unk>"
 }

   "pad_token": "<pad>",
   "sep_token": "</s>",
   "tokenizer_class": "XLMRobertaTokenizer",
+  "unk_token": "<unk>",
+  "truncation": true
 }