--- library_name: transformers license: apache-2.0 base_model: - microsoft/Phi-3-medium-128k-instruct datasets: - flammenai/FlameMix-DPO-v1 - flammenai/Grill-preprod-v1_chatML - flammenai/Grill-preprod-v2_chatML --- **Exllamav2** quant (**exl2** / **4.25 bpw**) made with ExLlamaV2 v0.0.21 Other EXL2 quants: | **Quant** | **Model Size** | **lm_head** | | ----- | ---------- | ------- | |
**[2.2](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-2_2bpw_exl2)**
|
4032 MB
|
6
| |
**[2.5](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-2_5bpw_exl2)**
|
4485 MB
|
6
| |
**[3.0](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-3_0bpw_exl2)**
|
5312 MB
|
6
| |
**[3.5](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-3_5bpw_exl2)**
|
6116 MB
|
6
| |
**[3.75](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-3_75bpw_exl2)**
|
6526 MB
|
6
| |
**[4.0](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-4_0bpw_exl2)**
|
6936 MB
|
6
| |
**[4.25](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-4_25bpw_exl2)**
|
7312 MB
|
6
| |
**[5.0](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-5_0bpw_exl2)**
|
8559 MB
|
6
| |
**[6.0](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-6_0bpw_exl2)**
|
10220 MB
|
8
| |
**[6.5](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-6_5bpw_exl2)**
|
11013 MB
|
8
| |
**[8.0](https://huggingface.co/Zoyd/flammenai_Mahou-1.2-phi-14B-8_0bpw_exl2)**
|
12726 MB
|
8
| ![image/png](https://huggingface.co/flammenai/Mahou-1.0-mistral-7B/resolve/main/mahou1.png) # Mahou-1.2-phi-14B Please note: this is an untested, experimental release. Mahou is our attempt to build a production-ready conversational/roleplay LLM. Future versions will be released iteratively and finetuned from flammen.ai conversational data. ### Chat Format This model has been trained to use ChatML format. ``` <|im_start|>system {{system}}<|im_end|> <|im_start|>{{char}} {{message}}<|im_end|> <|im_start|>{{user}} {{message}}<|im_end|> ``` # Roleplay Format - Speech without quotes. - Actions in `*asterisks*` ``` *leans against wall cooly* so like, i just casted a super strong spell at magician academy today, not gonna lie, felt badass. ``` ### ST Settings 1. Use ChatML for the Context Template. 2. Turn on Instruct Mode for ChatML. 3. Use the following stopping strings: `["<", "|", "<|", "\n"]` ### Method Finetuned using an A100 on Google Colab. [Fine-tune a Mistral-7b model with Direct Preference Optimization](https://towardsdatascience.com/fine-tune-a-mistral-7b-model-with-direct-preference-optimization-708042745aac) - [Maxime Labonne](https://huggingface.co/mlabonne) ### Configuration LoRA, model, and training settings: ```python # LoRA configuration peft_config = LoraConfig( r=16, lora_alpha=16, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM", target_modules=['k_proj', 'gate_proj', 'v_proj', 'up_proj', 'q_proj', 'o_proj', 'down_proj'] ) # Model to fine-tune model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=torch.bfloat16, load_in_4bit=True ) model.config.use_cache = False # Reference model ref_model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=torch.bfloat16, load_in_4bit=True ) # Training arguments training_args = TrainingArguments( per_device_train_batch_size=4, gradient_accumulation_steps=4, gradient_checkpointing=True, learning_rate=5e-5, lr_scheduler_type="cosine", max_steps=2000, save_strategy="no", logging_steps=1, output_dir=new_model, optim="paged_adamw_32bit", warmup_steps=100, bf16=True, report_to="wandb", ) # Create DPO trainer dpo_trainer = DPOTrainer( model, ref_model, args=training_args, train_dataset=dataset, tokenizer=tokenizer, peft_config=peft_config, beta=0.1, force_use_ref_model=True ) # Fine-tune model with DPO dpo_trainer.train() ```