Xenova HF staff commited on
Commit
d1b752a
1 Parent(s): fbcb28b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +70 -0
README.md CHANGED
@@ -6,4 +6,74 @@ tags:
6
 
7
  https://huggingface.co/laion/larger_clap_general with ONNX weights to be compatible with Transformers.js.
8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).
 
6
 
7
  https://huggingface.co/laion/larger_clap_general with ONNX weights to be compatible with Transformers.js.
8
 
9
+ ## Usage (Transformers.js)
10
+
11
+ If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@xenova/transformers) using:
12
+ ```bash
13
+ npm i @xenova/transformers
14
+ ```
15
+
16
+ **Example:** Perform zero-shot audio classification with `Xenova/larger_clap_general`.
17
+ ```js
18
+ import { pipeline } from '@xenova/transformers';
19
+
20
+ const classifier = await pipeline('zero-shot-audio-classification', 'Xenova/larger_clap_general');
21
+
22
+ const audio = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/piano.wav';
23
+ const candidate_labels = ['calm piano music', 'heavy metal music'];
24
+ const scores = await classifier(audio, candidate_labels);
25
+ // [
26
+ // { score: 0.9829504489898682, label: 'calm piano music' },
27
+ // { score: 0.017049523070454597, label: 'heavy metal music' }
28
+ // ]
29
+ ```
30
+
31
+ **Example:** Compute text embeddings with `ClapTextModelWithProjection`.
32
+
33
+ ```js
34
+ import { AutoTokenizer, ClapTextModelWithProjection } from '@xenova/transformers';
35
+
36
+ // Load tokenizer and text model
37
+ const tokenizer = await AutoTokenizer.from_pretrained('Xenova/larger_clap_general');
38
+ const text_model = await ClapTextModelWithProjection.from_pretrained('Xenova/larger_clap_general');
39
+
40
+ // Run tokenization
41
+ const texts = ['calm piano music', 'heavy metal music'];
42
+ const text_inputs = tokenizer(texts, { padding: true, truncation: true });
43
+
44
+ // Compute embeddings
45
+ const { text_embeds } = await text_model(text_inputs);
46
+ // Tensor {
47
+ // dims: [ 2, 512 ],
48
+ // type: 'float32',
49
+ // data: Float32Array(1024) [ ... ],
50
+ // size: 1024
51
+ // }
52
+ ```
53
+
54
+ **Example:** Compute audio embeddings with `ClapAudioModelWithProjection`.
55
+ ```js
56
+ import { AutoProcessor, ClapAudioModelWithProjection, read_audio } from '@xenova/transformers';
57
+
58
+ // Load processor and audio model
59
+ const processor = await AutoProcessor.from_pretrained('Xenova/larger_clap_general');
60
+ const audio_model = await ClapAudioModelWithProjection.from_pretrained('Xenova/larger_clap_general');
61
+
62
+ // Read audio and run processor
63
+ const audio = await read_audio('https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/piano.wav');
64
+ const audio_inputs = await processor(audio);
65
+
66
+ // Compute embeddings
67
+ const { audio_embeds } = await audio_model(audio_inputs);
68
+ // Tensor {
69
+ // dims: [ 1, 512 ],
70
+ // type: 'float32',
71
+ // data: Float32Array(512) [ ... ],
72
+ // size: 512
73
+ // }
74
+ ```
75
+
76
+ ---
77
+
78
+
79
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).