Qwen
/

Qwen2-Audio-7B-Instruct

Audio-Text-to-Text

text2text-generation

Inference Endpoints

Model card Files Files and versions Community

freddyaboulton HF staff commited on 25 days ago

Commit

9fffe91

•

1 Parent(s): bac62d2

Move input_ids to cuda device

The original code does not work on ZeroGPU

Files changed (1) hide show

README.md +3 -2

README.md CHANGED Viewed

@@ -65,10 +65,11 @@ for message in conversation:
                 )
 inputs = processor(text=text, audios=audios, return_tensors="pt", padding=True)
-inputs.input_ids = inputs.input_ids.to("cuda")
 generate_ids = model.generate(**inputs, max_length=256)
-generate_ids = generate_ids[:, inputs.input_ids.size(1):]
 response = processor.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
 ```

                 )
 inputs = processor(text=text, audios=audios, return_tensors="pt", padding=True)
+inputs = dict(**inputs)
+inputs["input_ids"] = inputs["input_ids"].to("cuda")
 generate_ids = model.generate(**inputs, max_length=256)
+generate_ids = generate_ids[:, inputs["input_ids"].size(1):]
 response = processor.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
 ```