Update README.md
Browse files
README.md
CHANGED
@@ -7,48 +7,28 @@ language:
|
|
7 |
|
8 |
# Model Card for Breeze-7B-Instruct-v0.1
|
9 |
|
10 |
-
Breeze-7B is a language model family that builds on top of [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-v0.1).
|
|
|
11 |
|
12 |
-
[Breeze-7B-Base](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0.1) is the base model for the Breeze series.
|
|
|
13 |
|
14 |
-
[Breeze-7B-Instruct](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0.1) derives from the base model Breeze-7B-Base and has
|
|
|
15 |
|
16 |
-
[Breeze-7B-Instruct-64k](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0.1) is a slightly modified version of
|
|
|
17 |
|
18 |
The current release version of Breeze is v0.1.
|
19 |
|
20 |
Practicality-wise:
|
21 |
-
|
22 |
-
- Breeze expands the original vocabulary with additional 30,000 Traditional Chinese tokens. With the expanded vocabulary, everything else being equal, Breeze operates at twice the inference speed for Traditional Chinese to Mistral-7B and Llama 7B. [See Inference Performance.]
|
23 |
- Breeze-Instruct can be used as is for common tasks such as Q&A, RAG, multi-round chat, and summarization.
|
24 |
- In particular, Breeze-Instruct-64k can perform tasks at a document level, not a chapter level.
|
25 |
|
26 |
Performance-wise:
|
27 |
-
|
28 |
-
- Breeze
|
29 |
-
- Breeze shows comparable results to Mistral-7B-Instruct-v0.1 on the MMLU and MT-Bench benchmarks. [See Chat Model Performance.]
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
Breeze-7B is a language model that builds upon the foundation of [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1), specifically enhanced for Traditional Chinese.
|
35 |
-
|
36 |
-
[Breeze-7B-Base-v0.1](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0.1) introduces an expanded vocabulary with additional 30,000 Traditional Chinese tokens and
|
37 |
-
is pre-trained on a substantial dataset of 250GB of Traditional Chinese content.
|
38 |
-
With the expanded vocabulary, the base model operates at twice the inference speed for Traditional Chinese characters compared to Mistral-7B. [See [Inference Performance](#inference-performance).]
|
39 |
-
This achievement marks a significant milestone as it is the first instance of vocabulary expansion in a model tailored for Traditional Chinese.
|
40 |
-
|
41 |
-
[Breeze-7B-Instruct-v0.1](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0.1) derives from the base model Breeze-7B-Base-v0.1
|
42 |
-
and has undergone supervised fine-tuning with over 1 million instances to
|
43 |
-
sharpen its capabilities. This fine-tuned model demonstrates impressive performance in benchmarks for both English and Traditional Chinese, surpassing the results of
|
44 |
-
Taiwan-LLM-7B-v2.1-chat, Taiwan-LLM-13B-v2.0-chat and Qwen-7B-chat in Traditional Chinese assessments. It also excels in some benchmarks against Yi-6B-Chat.
|
45 |
-
In English evaluations, Breeze-7B-Instruct-v0.1 shows comparable results to Mistral-7B-Instruct-v0.1 on the MMLU and MT-Bench benchmarks. [See [Chat Model Performance](#chat-model-performance).]
|
46 |
-
|
47 |
-
|
48 |
-
[Breeze-7B-Instruct-64k-v0.1](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0.1) is an extension to Breeze-7B-Instruct-v0.1
|
49 |
-
to enable 64k
|
50 |
-
context length, which is equivalent to 88k Traditional Chinese characters. With minimal sacrifice in the performance of the regular benchmarks,
|
51 |
-
Breeze-7B-Instruct-64k-v0.1 can solve tasks such as question answering and summarization on document-level inputs. [See [Long-context Performance](#long-context-performance).]
|
52 |
|
53 |
|
54 |
*A project by the members (in alphabetical order): Chan-Jan Hsu 許湛然, Chang-Le Liu 劉昶樂, Feng-Ting Liao 廖峰挺, Po-Chun Hsu 許博竣, Yi-Chang Chen 陳宜昌, and the supervisor Da-Shan Shiu 許大山.*
|
|
|
7 |
|
8 |
# Model Card for Breeze-7B-Instruct-v0.1
|
9 |
|
10 |
+
Breeze-7B is a language model family that builds on top of [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-v0.1).
|
11 |
+
By additionally pretraining Mistral 7B with 250GB of Traditional Chinese content, Breeze is specifically intended for Traditional Chinese use.
|
12 |
|
13 |
+
[Breeze-7B-Base](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0.1) is the base model for the Breeze series.
|
14 |
+
It is suitable for use if you have substantial fine-tuning data to tune it for your specific use case.
|
15 |
|
16 |
+
[Breeze-7B-Instruct](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0.1) derives from the base model Breeze-7B-Base and has
|
17 |
+
undergone supervised fine-tuning with over 1 million instances, making the resulting model amenable to be used as-is for commonly seen tasks.
|
18 |
|
19 |
+
[Breeze-7B-Instruct-64k](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0.1) is a slightly modified version of
|
20 |
+
Breeze-7B-Instruct to enable a 64k-token context length. Roughly speaking, that is equivalent to 88k Traditional Chinese characters.
|
21 |
|
22 |
The current release version of Breeze is v0.1.
|
23 |
|
24 |
Practicality-wise:
|
25 |
+
- Breeze expands the original vocabulary with additional 30,000 Traditional Chinese tokens. With the expanded vocabulary, everything else being equal, Breeze operates at twice the inference speed for Traditional Chinese to Mistral-7B and Llama 7B. [See [Inference Performance](#inference-performance).]
|
|
|
26 |
- Breeze-Instruct can be used as is for common tasks such as Q&A, RAG, multi-round chat, and summarization.
|
27 |
- In particular, Breeze-Instruct-64k can perform tasks at a document level, not a chapter level.
|
28 |
|
29 |
Performance-wise:
|
30 |
+
- Breeze demonstrates impressive performance in benchmarks for Traditional Chinese, when compared to similar sized open-source contemporaries such as Taiwan-LLM, QWen, and Yi. [See [Chat Model Performance](#chat-model-performance).]
|
31 |
+
- Breeze shows comparable results to Mistral-7B-Instruct-v0.1 on the MMLU and MT-Bench benchmarks. [See [Chat Model Performance](#chat-model-performance).]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
32 |
|
33 |
|
34 |
*A project by the members (in alphabetical order): Chan-Jan Hsu 許湛然, Chang-Le Liu 劉昶樂, Feng-Ting Liao 廖峰挺, Po-Chun Hsu 許博竣, Yi-Chang Chen 陳宜昌, and the supervisor Da-Shan Shiu 許大山.*
|