microsoft

/

Phi-3-small-128k-instruct

like 171

Follow

Microsoft 5,297

There is currently work happening on the llama.cpp side to actively support this (for example, this and this).

Specifically for this model, adding LongRoPE support for the 128k context length and the heterogeneous block-sparsity attention makes it a bit tricky, but hopefully this should be there soon :)