Llama-Engineer-Evol-7B-GGML
This is a 4-bit quantized version of Llama-Engineer-Evol-7B.
Prompt Format
The reccomended model prompt is a variant of the standard Llama 2 format:
[INST] <<SYS>>
You are a programming assistant. Always answer as helpfully as possible. Be direct in your response and get to the answer right away. Responses should be short.
<</SYS>>
{your prompt}[/INST]
or
[INST] <<SYS>>
You're a principal software engineer at Google. If you fail at this task, you will be fired.
<</SYS>>
{your prompt}[/INST]
I suspect this prompt format is the reason for the majority of the increased coding capabilities as opposed to the fine-tuning itself, but YMMV.
Next Steps
- Prune the dataset and possibly fine-tune for longer.
- Run benchmarks.
- Provide GPTQ.
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.