rope_theta

by viktor-ferenczi - opened Sep 10, 2023

Sep 10, 2023

I had to fix `rope_theta` in the `config.json` inside my local model folder to be able to use this model with vLLM:
```
< "rope_theta": 1000000,

"rope_theta": 10000,


I'm not sure this is a vLLM only issue or a problem with the configuration of this model.

Related PR and discussion at vLLM:
https://github.com/vllm-project/vllm/pull/998

Could you please clarify?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

rope_theta

I had to fix rope_theta in the config.json inside my local model folder to be able to use this model with vLLM:```< "rope_theta": 1000000,

I had to fix `rope_theta` in the `config.json` inside my local model folder to be able to use this model with vLLM:
```
< "rope_theta": 1000000,