Baicai003's picture
Update README.md
23a3cf3
|
raw
history blame
No virus
4.85 kB
---
library_name: transformers
datasets:
- shareAI/ShareGPT-Chinese-English-90k
- FreedomIntelligence/ShareGPT-CN
language:
- zh
pipeline_tag: question-answering
tags:
- chat
- llm
- llama2
- chatgpt
---
- Github:https://github.com/CrazyBoyM/llama2-Chinese-chat
更新:
- 2023-7-19 首个llama2 13b中文对话版本放出。
- 2023-07-23 完成第2个epoch训练放出,测试有更好的对话体验
- 2023-08-03 分支版本:bimoGPT放出,拥有自我身份认知、不错的代码问答能力,下载地址:https://huggingface.co/shareAI/bimoGPT-llama2-13b
- 2023-08-21 更新世界模型排名榜,超越某号称“中文Llama2官方”社区的收费模型十多个名次。
完整合并后文件下载:https://www.codewithgpu.com/m/file/llama2-13b-Chinese-chat
- 训练用数据集:https://huggingface.co/datasets/shareAI/ShareGPT-Chinese-English-90k
- llama2训练交流QQ群:443064756
项目在中文sharegpt数据集上训练得到的llama2 Chinese chat 13b,为减轻文件大小负担这里只放出了adapter的权重
请拉取https://huggingface.co/TheBloke/Llama-2-13B-fp16 作为基础权重,使用如下脚步执行合并得到可工作的总权重:
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name_or_path = '/data/TheBloke/Llama-2-13B-fp16'
adapter_name_or_path = '/data/llama2-13b-Chinese-chat'
save_path = '/data/llama2-13b-Chinese-chat_v1'
tokenizer = AutoTokenizer.from_pretrained(
model_name_or_path,
trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
model_name_or_path,
trust_remote_code=True,
low_cpu_mem_usage=True,
torch_dtype=torch.float16,
device_map='auto'
)
print("load model success")
model = PeftModel.from_pretrained(model, adapter_name_or_path)
print("load adapter success")
model = model.merge_and_unload()
print("merge success")
tokenizer.save_pretrained(save_path)
model.save_pretrained(save_path)
print("save done.")
```
合并后,体验对话:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
def main():
model_name = '/data/llama2-13b-Chinese-chat_v1'
device = 'cuda'
max_new_tokens = 500 # 每轮对话最多生成多少个token
history_max_len = 2000 # 模型记忆的最大token长度
top_p = 0.9
temperature = 0.35 # 越大模型越浪
repetition_penalty = 1.2 # 如果模型出现重复说话可以调节该系数
# 加载模型
model = AutoModelForCausalLM.from_pretrained(
model_name,
trust_remote_code=True,
low_cpu_mem_usage=True,
torch_dtype=torch.float16,
device_map='auto'
).to(device).eval()
tokenizer = AutoTokenizer.from_pretrained(
model_name,
trust_remote_code=True,
# llama不支持fast
use_fast=False if model.config.model_type == 'llama' else True
)
# 记录所有历史记录
history_token_ids = tokenizer('<s>', return_tensors="pt").input_ids
# 开始对话
user_input = input('User:')
while True:
user_input = '{}</s>'.format(user_input)
user_input_ids = tokenizer(user_input, return_tensors="pt", add_special_tokens=False).input_ids
history_token_ids = torch.concat((history_token_ids, user_input_ids), dim=1)
model_input_ids = history_token_ids[:, -history_max_len:].to(device)
with torch.no_grad():
outputs = model.generate(
input_ids=model_input_ids, max_new_tokens=max_new_tokens, do_sample=True, top_p=top_p,
temperature=temperature, repetition_penalty=repetition_penalty, eos_token_id=tokenizer.eos_token_id
)
model_input_ids_len = model_input_ids.size(1)
response_ids = outputs[:, model_input_ids_len:]
history_token_ids = torch.concat((history_token_ids, response_ids.cpu()), dim=1)
response = tokenizer.batch_decode(response_ids)
print("Bot:" + response[0].strip().replace('</s>', ""))
user_input = input('User:')
if __name__ == '__main__':
main()
```
推荐继续二次训练以针对性调优对话效果~
## Training procedure
The following `bitsandbytes` quantization config was used during training:
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: float16
### Framework versions
- PEFT 0.4.0.dev0
训练1个epoch,loss 0.9,实测用中文对话体验优于baichuan13b(仅主观感受)。还有很大潜力,建议作为底座把文件拉回去继续调优。
感谢:
- LLaMA2
- Firefly项目
- shareGPT中文数据集的建设者们