|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- stingning/ultrachat |
|
- kaist-ai/CoT-Collection |
|
- mesolitica/google-translate-commitpackft |
|
- Wanfq/Explore_Instruct_Rewriting_32k |
|
- Wanfq/Explore_Instruct_Rewriting_10k |
|
- Wanfq/Explore_Instruct_Brainstorming_16k |
|
- xiyuez/red-dot-design-award-product-description |
|
--- |
|
|
|
# RWKV v4 7B world model |
|
finetuned with ultrachat , COT and some novel instructions data, commitpackft and so on |
|
|
|
use full ultrachat and cot data, about 3B tokens |
|
|
|
if you wanna do Role play, use [this model](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/RWKV-world-novel-one-state-ultrachat-cot-tuned-Role-play-65k.pth) |
|
|
|
|
|
# Contributor |
|
[@JL-er](https://huggingface.co/JL-er) |
|
[@Remixa](https://huggingface.co/Remixa) |
|
|
|
|
|
# Design of experiment |
|
this model lose multi-turn chat ability,cause from using whole ultrachat datasets. |
|
|
|
so i continue tuned multi-turn datasets with 2 aspects |
|
|
|
1.[role play](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/RWKV-world-novel-one-state-ultrachat-cot-tuned-Role-play-65k.pth) |
|
|
|
2.[novel multiturn instruction](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/rwkv-world-one-novel-cot-ultrachat-novel-instructions.pth) |
|
|
|
# Training details |
|
[wandb.ai](https://wandb.ai/one-/one-rwkv-64k) |
|
|
|
# CAses |
|
|
|
 |
|
|
|
 |
|
|
|
# Usage |
|
adjust tempp and topp on different scenario. |
|
 |
|
|
|
 |
|
|
|
 |
|
|
|
# COT and lookback |
|
|
|
 |
|
[this model](https://huggingface.co/xiaol/RWKV-4-world-one-state-novel-tuned-65k) can do above task with 100% acc. |
|
|
|
 |
|
|
|
## role play model |
|
|
|
 |
|
|
|
 |
|
|
|
## novel |
|
|
|
 |
|
|
|
 |
|
# demo site(temporary) |
|
[online showcase](https://rwkv.ai-creator.net/risu) |
