Hugging Face's logo

xiaol
/
RWKV-4-world-one-state-ultrachat-COT-65k

Model card Files Community
1
RWKV-4-world-one-state-ultrachat-COT-65k
File size: 2,942 Bytes 
cb20959
 
0420ce7
 
 
28deb88
 
 
 
 
892b20e
 
db2d4cc
b32d7c4
 
 
db2d4cc
061a077
 
892b20e
 
 
 
 
147ea7c
fec3606
 
147ea7c
fec3606
147ea7c
061a077
9c9fbd1
d3fd564
fec3606
892b20e
4e1b7fe
 
d2a9f77
 
 
 
 
 
4e1b7fe
62f63fb
4e1b7fe
1b4c23c
62f63fb
 
 
 
 
a76d32d
7f8c5be
 
6f2ef8f
62f63fb
 
6f2ef8f
 
 
 
62f63fb
 
 
 
 
 
ec62bf5
8c144d4
6f2ef8f
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
 
---
license: apache-2.0
datasets:
- stingning/ultrachat
- kaist-ai/CoT-Collection
- mesolitica/google-translate-commitpackft
- Wanfq/Explore_Instruct_Rewriting_32k
- Wanfq/Explore_Instruct_Rewriting_10k
- Wanfq/Explore_Instruct_Brainstorming_16k
- xiyuez/red-dot-design-award-product-description
---

# RWKV v4 7B world model 
finetuned with ultrachat , COT and some novel instructions data, commitpackft and so on

use full ultrachat and cot data, about 3B tokens

if you wanna do Role play, use [this model](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/RWKV-world-novel-one-state-ultrachat-cot-tuned-Role-play-65k.pth)


# Contributor
[@JL-er](https://huggingface.co/JL-er) 
[@Remixa](https://huggingface.co/Remixa)


# Design of experiment
this model lose multi-turn chat ability,cause from using whole ultrachat datasets.

so i continue tuned multi-turn datasets with 2 aspects

1.[role play](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/RWKV-world-novel-one-state-ultrachat-cot-tuned-Role-play-65k.pth)

2.[novel multiturn instruction](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/rwkv-world-one-novel-cot-ultrachat-novel-instructions.pth)

# Training details
[wandb.ai](https://wandb.ai/one-/one-rwkv-64k)

# CAses

![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/_1dJo549ldgX6q0JUwC6c.jpeg)

![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/7969wbHaJpBq2n6xvfC7C.jpeg)

# Usage
adjust tempp and topp on different scenario.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/cGDF6b4-x_9rcwMdl1KPp.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/_QduMpcGkCoC00DTlXQop.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/9CqrfEpcJcxtLoX5ffEFo.png)

# COT and lookback

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/hUxTVgjLBMcFqxQX9HoxL.png)
[this model](https://huggingface.co/xiaol/RWKV-4-world-one-state-novel-tuned-65k) can do above task with 100% acc.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/mXyfoD1_jlNqBElQfn_Fk.png)

## role play model

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/2ns794U56M1592w6Uy1dk.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/6CQp2kh56FFa-TmNUrld3.png)

## novel

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/wPXTYWFWfi-mPmoPbmfWg.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/ZgRK8N6jnxR3HawvFiUve.png)
# demo site(temporary)
[online showcase](https://rwkv.ai-creator.net/risu)