Add transformers.js support

#7
by Xenova HF staff - opened
Files changed (1) hide show
  1. README.md +31 -0
README.md CHANGED
@@ -7,6 +7,7 @@ tags:
7
  - mteb
8
  - arctic
9
  - snowflake-arctic-embed
 
10
  model-index:
11
  - name: snowflake-snowflake-arctic-embed-xs
12
  results:
@@ -3010,6 +3011,36 @@ for query, query_scores in zip(queries, scores):
3010
  print(score, document)
3011
  ```
3012
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3013
 
3014
  ## FAQ
3015
 
 
7
  - mteb
8
  - arctic
9
  - snowflake-arctic-embed
10
+ - transformers.js
11
  model-index:
12
  - name: snowflake-snowflake-arctic-embed-xs
13
  results:
 
3011
  print(score, document)
3012
  ```
3013
 
3014
+ ### Using Transformers.js
3015
+
3016
+ If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@xenova/transformers) by running:
3017
+ ```bash
3018
+ npm i @xenova/transformers
3019
+ ```
3020
+
3021
+ You can then use the model to compute embeddings as follows:
3022
+
3023
+ ```js
3024
+ import { pipeline, dot } from '@xenova/transformers';
3025
+
3026
+ // Create feature extraction pipeline
3027
+ const extractor = await pipeline('feature-extraction', 'Snowflake/snowflake-arctic-embed-xs', {
3028
+ quantized: false, // Comment out this line to use the quantized version
3029
+ });
3030
+
3031
+ // Generate sentence embeddings
3032
+ const sentences = [
3033
+ 'Represent this sentence for searching relevant passages: Where can I get the best tacos?',
3034
+ 'The Data Cloud!',
3035
+ 'Mexico City of Course!',
3036
+ ]
3037
+ const output = await extractor(sentences, { normalize: true, pooling: 'cls' });
3038
+
3039
+ // Compute similarity scores
3040
+ const [source_embeddings, ...document_embeddings ] = output.tolist();
3041
+ const similarities = document_embeddings.map(x => dot(source_embeddings, x));
3042
+ console.log(similarities); // [0.5044895661144148, 0.5636021124426508]
3043
+ ```
3044
 
3045
  ## FAQ
3046