support vllm

#10
by CarrotAI - opened

hello upstage
Thank you for sharing the model.

To make it easier for me to use it, I asked vllm for PR by referring to the given code. Can you explain the difference with Llama in more detail? And will the structure of the Pro model be changed in the future?

https://github.com/vllm-project/vllm/pull/8386

Yes please!

upstage org

@CarrotAI
Thank you for your interest!
The architectural difference from Llama is the presence of the BSKCN (Block level SKip CoNnection). The rope scaling mentioned in the PR is not different from Llama. We will officially review your PR and offer assistance in case any challenges or difficulties arise.

Thank you. It's been merged.

CarrotAI changed discussion status to closed

Sign up or log in to comment