What does the fim format request look like ?
Thank you very much for contributing this model. However, when I use the prompt "[PREFIX]def add([SUFFIX] return sum [MIDDLE]" in vllm, it does not return the result I expected. I look forward to your reply.
[PREFIX]def add( [SUFFIX] return sum [MIDDLE]
fixed
FIM has the following syntax:
[SUFFIX]{{{suffix}}}[PREFIX]{{{prefix}}}
I recommend you to first use the [SUFFIX], then the [PREFIX]. MIDDLE seems not really to be used.
I rely here on continues code snippet on how it is implemented in their codebase:
const codestralFimTemplate: AutocompleteTemplate = {
template: "[SUFFIX]{{{suffix}}}[PREFIX]{{{prefix}}}",
completionOptions: {
stop: ["[PREFIX]", "[SUFFIX]"],
},
};
(https://github.com/continuedev/continue/blob/029dc5aceac1df7cbde5f955f693a86c647eab61/core/autocomplete/templates.ts#L46)
Also for the instruction template, please see here:
https://github.com/ollama/ollama/issues/5403