Hugging Face's logo

01-ai
/
Yi-34B

Text Generation
Transformers
PyTorch
Safetensors
llama
text-generation-inference
Inference Endpoints
Model card Files Community
49
Yi-34B
File size: 4,971 Bytes 
1281cd3
 
 
 
 
994cf06
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
 
---
license: other
license_name: yi-license
license_link: LICENSE
---
<div align="center">

<h1>
  Yi
</h1>

<a href="https://github.com/01-ai/Yi/actions/workflows/ci.yml">
  <img src="https://github.com/01-ai/Yi/actions/workflows/ci.yml/badge.svg">
</a>
<a href="https://huggingface.co/01-ai">
  <img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-01--ai-blue">
</a>
<a href="https://www.modelscope.cn/organization/01ai/">
  <img src="https://img.shields.io/badge/ModelScope-01--ai-blue">
</a>
<a href="https://github.com/01-ai/Yi/blob/main/LICENSE">
  <img src="https://img.shields.io/github/license/01-ai/yi">
</a>

</div>

## Introduction

The **Yi** series models are large language models trained from scratch by developers at [01.AI](https://01.ai/). The first public release contains two base models with the parameter size of 6B and 34B. Besides, a specialized version with **200K** context window size is also provided.

## News

- 🎯 **2023/11/05**: The base model of `Yi-6B` and `Yi-34B` (with **200K** context window)

## Model Performance

(FIXME)

## Usage

### 1. Download the model (optional)

By default the model weights and tokenizer will be downloaded from [HuggingFace](https://huggingface.co/01-ai) automatically in the next step. You can also download them manually from the following places:

- ModelScope (FIXME)
- Mirror site (remember to extra the content with `tar`)
  - [Yi-6B.tar](https://01-ai.tos-cn-beijing.volces.com/yi/models/Yi-6B.tar)
  - [Yi-34B.tar](https://01-ai.tos-cn-beijing.volces.com/yi/models/Yi-34B.tar)

### 2. Run with docker

The recommended approach to try out our models is through docker. We provide the following docker images.

- `ghcr.io/01-ai/yi:latest`
- `ml-a100-cn-beijing.cr.volces.com/ci/01-ai/yi:latest`

Note that the `latest` tag always point to the latest code in the `main` branch. To test a stable version, please replace it with a specific [tag](https://github.com/01-ai/Yi/tags).

#### 2.1 Try out the base model:

```bash
docker run -it ghcr.io/01-ai/yi:latest python demo/text_generation.py
```

To reuse the downloaded models in the previous step, you can mount them into the container:

```bash
docker run -it                   \
  -v /path/to/model:/model       \
  ghcr.io/01-ai/yi:latest        \
  python demo/text_generation.py \
  --model /model
```

For more advanced usage, please refer the [doc](./demo/README.md).

#### 2.2 Finetuning from the base model:

```bash
docker run -it                                      \
  -v /path/to/base/model:/base_model                \
  -v /path/to/save/finetuned/model:/finetuned_model \
  ghcr.io/01-ai/yi:latest                           \
  bash finetune/scripts/run_sft_Yi_6b.sh
```

Once finished, you can compare the finetuned model and the base model with the following command:

```bash
docker run -it                                       \
  -v /path/to/save/finetuned/model/:/finetuned_model \
  -v /path/to/base/model/:/base_model                \
  ghcr.io/01-ai/yi:latest                            \
  bash finetune/scripts/run_eval.sh
```

For more advanced usage like fine-tuning based on your custom data, please refer the [doc](./finetune/README.md).

#### 2.3 Quantization

```bash
docker run -it                                         \
  -v /path/to/base/model:/base_model                   \
  -v /path/to/save/quantization/model:/quantized_model \
  ghcr.io/01-ai/yi:latest                              \
  python quantization/gptq/quant_autogptq.py           \
  --model /base_model                                  \
  --output_dir /quantized_model                        \
  --trust_remote_code        
```

Once finished, you can then evaluate the resulted model as follows:

```bash
docker run -it                                         \
  -v /path/to/save/quantization/model:/quantized_model \
  ghcr.io/01-ai/yi:latest                              \
  python quantization/gptq/eval_quantized_model.py     \
  --model /quantized_model                             \
  --trust_remote_code
```

For more detailed explanation, please read the [doc](./quantization/gptq/README.md)

## Disclaimer

Although we use data compliance checking algorithms during the training process to ensure the compliance of the trained model to the best of our ability, due to the complexity of the data and the diversity of language model usage scenarios, we cannot guarantee that the model will generate correct and reasonable output in all scenarios. Please be aware that there is still a risk of the model producing problematic outputs. We will not be responsible for any risks and issues resulting from misuse, misguidance, illegal usage, and related misinformation, as well as any associated data security concerns.

## License

The source code in this repo is licensed under the [Apache 2.0 license](./LICENSE). The Yi series model must be adhere to the [Model License Agreement](./MODEL_LICENSE_AGREEMENT.txt).