README.md

---
datasets:
- THUDM/LongWriter-6k
language:
- en
- zh
library_name: transformers
license: llama3.1
tags:
- Long Context
- chatglm
- llama
- mlx
---

# shi3z/mlx-LongWriter-llama3.1-8b-8bit

The Model [shi3z/mlx-LongWriter-llama3.1-8b-8bit](https://huggingface.co/shi3z/mlx-LongWriter-llama3.1-8b-8bit) was converted to MLX format from [THUDM/LongWriter-llama3.1-8b](https://huggingface.co/THUDM/LongWriter-llama3.1-8b) using mlx-lm version **0.17.0**.

## Use with mlx

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("shi3z/mlx-LongWriter-llama3.1-8b-8bit")
response = generate(model, tokenizer, prompt="hello", verbose=True)
```