---
library_name: transformers
language:
- ru
- uk
- kk
- be
---

## About model creation

This is a smaller version of the **intfloat/multilingual-e5-base** with only some Russian (Cyrillic in general) and English (fever) tokens (and embeddings) left.

The model created in a similar way as described in this https://medium.com/m/global-identity-2?redirectUrl=https%3A%2F%2Ftowardsdatascience.com%2Fhow-to-adapt-a-multilingual-t5-model-for-a-single-language-b9f94f3d9c90 post. 

The **CulturaX** dataset was used to search for the required tokens. As a result, out of 250k tokens of the original model, only **69,382** required were left.

## Was the model trained in any way?

No. The tokenizer has been modified, and all changes to token identifiers have been corrected by moving embeddings in the model word_embeddings module to their new places, so **the quality of this model** on Cyrilic (and English) **is exactly the same** as the original one.

## Why do we need this?

This allows you to use significantly less memory during training and also greatly reduces the weight of the model.

## Authors
- Sergei Bratchikov (https://t.me/nlpwanderer)