---
language:
- ms
---

# QLORA Malaysian Llama2 7B 32k chat completions

QLORA https://huggingface.co/mesolitica/llama-7b-hf-32768-fpf on translated UltraChat, https://huggingface.co/datasets/mesolitica/google-translate-ultrachat. 

We use exact Llama2 chat template.

README at https://github.com/mesolitica/malaya/tree/5.1/session/llama2#7b-16384-context-length-flash-attention-2