FAQ

Installation & Environment

Which version of transformers should I use?

4.31.0 is preferred.

I downloaded the codes and checkpoints but I can't load the model locally. What should I do?

Please check if you have updated the code to the latest, and correctly downloaded all the sharded checkpoint files.

`qwen.tiktoken` is not found. What is it?

This is the merge file of the tokenizer. You have to download it. Note that if you just git clone the repo without git-lfs, you cannot download this file.

transformers_stream_generator/tiktoken/accelerate not found

Run the command pip install -r requirements.txt. You can find the file at https://github.com/QwenLM/Qwen-VL/blob/main/requirements.txt.

Demo & Inference

Is there any demo?

Yes, see web_demo_mm.py for web demo. See README for more information.

Can Qwen-VL support streaming?

No. We do not support streaming yet.

It seems that the generation is not related to the instruction...

Please check if you are loading Qwen-VL-Chat instead of Qwen-VL. Qwen-VL is the base model without alignment, which behaves differently from the SFT/Chat model.

Is quantization supported?

No. We would support quantization asap.

Unsatisfactory performance in processing long sequences

Please ensure that NTK is applied. use_dynamc_ntk and use_logn_attn in config.json should be set to true (true by default).

Tokenizer

bos_id/eos_id/pad_id not found

In our training, we only use <|endoftext|> as the separator and padding token. You can set bos_id, eos_id, and pad_id to tokenizer.eod_id. Learn more about our tokenizer from our documents about the tokenizer.

FAQ