Xenova HF staff commited on
Commit
214aa9f
1 Parent(s): 187b05e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -2
README.md CHANGED
@@ -13,8 +13,45 @@ If you haven't already, you can install the [Transformers.js](https://huggingfac
13
  npm i @huggingface/transformers
14
  ```
15
 
16
- **Example:** TODO
17
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
 
19
  ## ONNX conversion script:
20
  First, install the following dependencies:
 
13
  npm i @huggingface/transformers
14
  ```
15
 
16
+ **Example:** Image+text to text
17
+
18
+ ```js
19
+ import { AutoProcessor, Qwen2VLForConditionalGeneration, RawImage } from "@huggingface/transformers";
20
+
21
+ // Load processor and model
22
+ const model_id = "onnx-community/Qwen2-VL-2B-Instruct";
23
+ const processor = await AutoProcessor.from_pretrained(model_id);
24
+ const model = await Qwen2VLForConditionalGeneration.from_pretrained(model_id);
25
+
26
+ // Prepare inputs
27
+ const url = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg";
28
+ const image = await (await RawImage.read(url)).resize(448, 448);
29
+ const conversation = [
30
+ {
31
+ role: "user",
32
+ content: [
33
+ { type: "image" },
34
+ { type: "text", text: "Describe this image." },
35
+ ],
36
+ },
37
+ ];
38
+ const text = processor.apply_chat_template(conversation, { add_generation_prompt: true });
39
+ const inputs = await processor(text, image);
40
+
41
+ // Perform inference
42
+ const outputs = await model.generate({
43
+ ...inputs,
44
+ max_new_tokens: 128,
45
+ });
46
+
47
+ // Decode output
48
+ const decoded = processor.batch_decode(
49
+ outputs.slice(null, [inputs.input_ids.dims.at(-1), null]),
50
+ { skip_special_tokens: true },
51
+ );
52
+ console.log(decoded[0]);
53
+ // The image depicts a serene beach scene with a woman and a dog. The woman is sitting on the sand, wearing a plaid shirt, and appears to be engaged in a playful interaction with the dog. The dog, which is a large breed, is sitting on its hind legs and appears to be reaching out to the woman, possibly to give her a high-five or a paw. The background shows the ocean with gentle waves, and the sky is clear, suggesting it might be either sunrise or sunset. The overall atmosphere is calm and relaxed, capturing a moment of connection between the woman and the dog.
54
+ ```
55
 
56
  ## ONNX conversion script:
57
  First, install the following dependencies: