File size: 3,071 Bytes
14a6663 9eeca79 ad17380 14a6663 dabb348 8899427 cf6606d 83bfb42 cf6606d d5a624d cf6606d d5a624d cf6606d 9eeca79 c8a6ce5 186b105 c8a6ce5 186b105 c8a6ce5 cf6606d 9eeca79 3b900c3 7ceb137 3b900c3 6d60562 2fcc6a7 de375bb 09dbf8c 83bfb42 cf6606d b267cfb |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 |
---
license: llama2
datasets:
- rombodawg/LosslessMegaCodeTrainingV2_1m_Evol_Uncensored
---
___________________________
- Please note this model was not trained on the rombodawg/LosslessMegaCodeTrainingV3_MINI dataset, despite the name similarity. You can find the training data at the bottom of the model card labeled (megacode2-min100)
___________________________
This is one of the first models trained on the LosslessMegaCodeTrainingV2_1m_Evol_Uncensored dataset. The version of the dataset used for this model was filtered by removed any data with less than 100 tokens but plans for much more refined filtering are in the works
- This model was made as a colaboration between me and andreaskoepf who is an affiliate of Open Assistant.
This model is extremely good at coding, and might be one of the best coding models for its size and much better than any 7b parameter model. Plans for bigger models are coming in the future.
### Prompt template
[chatml](https://github.com/openai/openai-python/blob/main/chatml.md) format is used:
"<|im_start|>system\n{system message}<|im_end|>\n<|im_start|>user\n{user prompt}<|im_end|>\n<|im_start|>assistant\n{Assistant answer}<|im_end|>\n"
multi-line:
```
<|im_start|>system
{system message}<|im_end|>
<|im_start|>user
{user prompt}<|im_end|>
<|im_start|>assistant
{Assistant answer}<|im_end|>
```
Gpt4all template:
- System prompt
```
<|im_start|>system
"Below is an instruction that describes a task. Write a response that appropriately completes the request."
```
- Prompt template
```
<|im_end|>
<|im_start|>user
"%1"<|im_end|>
<|im_start|>assistant
```
Oobagooba Text-Generation-Webui Template
- user:
```
<|im_start|>user
{User string}<|im_end|>
```
- bot:
```
<|im_start|>assistant
{Bot string}<|im_end|>
```
- turn_template:
```
<|user|>\n<|user-message|>\n\n<|bot|>\n<|bot-message|>\n\n
```
- context:
```
<|im_start|>system
Below is an instruction that describes a task. Write a response that appropriately completes the request.<|im_end|>
```
Current quatizations available:
- https://huggingface.co/TheBloke/LosslessMegaCoder-Llama2-7B-Mini-GPTQ
Benchmarks for the model can be found at the link bellow the model here is called (andreaskoepf/llama2-7b-megacode2_min100)
- https://tju01.github.io/FastEval-OpenAssistant/
Sampling report:
https://open-assistant.github.io/oasst-model-eval/?f=https%3A%2F%2Fraw.githubusercontent.com%2FOpen-Assistant%2Foasst-model-eval%2Fmain%2Fsampling_reports%2Foasst-pretrained%2F2023-08-12_andreaskoepf_llama2-7b-megacode2_min100_sampling_noprefix2.json
Training information:
- https://wandb.ai/open-assistant/public-sft/runs/run17_megacode_min100
The link for the full dataset is bellow:
- https://huggingface.co/datasets/rombodawg/LosslessMegaCodeTrainingV2_1m_Evol_Uncensored
Link for the filtered dataset used to make this model are bellow:
- https://huggingface.co/datasets/andreaskoepf/megacode2-min100
The original posting for this model was uploaded at the link bellow.
- https://huggingface.co/andreaskoepf/llama2-7b-megacode2_min100 |