Add support for Transformers.js

#5
by Xenova HF staff - opened
Files changed (1) hide show
  1. README.md +32 -0
README.md CHANGED
@@ -8,6 +8,7 @@ tags:
8
  - mteb
9
  - arctic
10
  - snowflake-arctic-embed
 
11
  model-index:
12
  - name: snowflake-arctic-embed-l
13
  results:
@@ -3013,6 +3014,37 @@ for query, query_scores in zip(queries, scores):
3013
  print(score, document)
3014
  ```
3015
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3016
 
3017
  ## FAQ
3018
 
 
8
  - mteb
9
  - arctic
10
  - snowflake-arctic-embed
11
+ - transformers.js
12
  model-index:
13
  - name: snowflake-arctic-embed-l
14
  results:
 
3014
  print(score, document)
3015
  ```
3016
 
3017
+ ### Using Transformers.js
3018
+
3019
+ If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@xenova/transformers) by running:
3020
+ ```bash
3021
+ npm i @xenova/transformers
3022
+ ```
3023
+
3024
+ You can then use the model to compute embeddings as follows:
3025
+
3026
+ ```js
3027
+ import { pipeline, dot } from '@xenova/transformers';
3028
+
3029
+ // Create feature extraction pipeline
3030
+ const extractor = await pipeline('feature-extraction', 'Snowflake/snowflake-arctic-embed-l', {
3031
+ quantized: false, // Comment out this line to use the quantized version
3032
+ });
3033
+
3034
+ // Generate sentence embeddings
3035
+ const sentences = [
3036
+ 'Represent this sentence for searching relevant passages: Where can I get the best tacos?',
3037
+ 'The Data Cloud!',
3038
+ 'Mexico City of Course!',
3039
+ ]
3040
+ const output = await extractor(sentences, { normalize: true, pooling: 'cls' });
3041
+
3042
+ // Compute similarity scores
3043
+ const [source_embeddings, ...document_embeddings ] = output.tolist();
3044
+ const similarities = document_embeddings.map(x => dot(source_embeddings, x));
3045
+ console.log(similarities); // [0.25145517380846977, 0.3865060421197194]
3046
+ ```
3047
+
3048
 
3049
  ## FAQ
3050