Add transformers.js support

#5
by Xenova HF staff - opened
Files changed (1) hide show
  1. README.md +32 -0
README.md CHANGED
@@ -7,6 +7,7 @@ tags:
7
  - mteb
8
  - arctic
9
  - snowflake-arctic-embed
 
10
  model-index:
11
  - name: snowflake-snowflake-arctic-embed-s
12
  results:
@@ -3012,6 +3013,37 @@ for query, query_scores in zip(queries, scores):
3012
  print(score, document)
3013
  ```
3014
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3015
  ## FAQ
3016
 
3017
 
 
7
  - mteb
8
  - arctic
9
  - snowflake-arctic-embed
10
+ - transformers.js
11
  model-index:
12
  - name: snowflake-snowflake-arctic-embed-s
13
  results:
 
3013
  print(score, document)
3014
  ```
3015
 
3016
+ ### Using Transformers.js
3017
+
3018
+ If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@xenova/transformers) by running:
3019
+ ```bash
3020
+ npm i @xenova/transformers
3021
+ ```
3022
+
3023
+ You can then use the model to compute embeddings as follows:
3024
+
3025
+ ```js
3026
+ import { pipeline, dot } from '@xenova/transformers';
3027
+
3028
+ // Create feature extraction pipeline
3029
+ const extractor = await pipeline('feature-extraction', 'Snowflake/snowflake-arctic-embed-s', {
3030
+ quantized: false, // Comment out this line to use the quantized version
3031
+ });
3032
+
3033
+ // Generate sentence embeddings
3034
+ const sentences = [
3035
+ 'Represent this sentence for searching relevant passages: Where can I get the best tacos?',
3036
+ 'The Data Cloud!',
3037
+ 'Mexico City of Course!',
3038
+ ]
3039
+ const output = await extractor(sentences, { normalize: true, pooling: 'cls' });
3040
+
3041
+ // Compute similarity scores
3042
+ const [source_embeddings, ...document_embeddings ] = output.tolist();
3043
+ const similarities = document_embeddings.map(x => dot(source_embeddings, x));
3044
+ console.log(similarities); // [0.48255123876493394, 0.5659250100112143]
3045
+ ```
3046
+
3047
  ## FAQ
3048
 
3049