Qwen2-72B-Instruct max_position_embedding is 32768?
#3
by
cizhenshi
- opened
I noticed that the blog mentions Qwen2-72B as a 128k model, but why is it configured as 32k in the settings?
Hi, to support longer context, extra configuration is required. Please refer to the model card: https://huggingface.co/Qwen/Qwen2-72B-Instruct#processing-long-texts
jklj077
changed discussion status to
closed