sho-takase
commited on
Commit
•
67facba
1
Parent(s):
a5e3187
Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,9 @@ This repository provides Japanese language models trained by [SB Intuitions](htt
|
|
11 |
|
12 |
## How to use
|
13 |
|
14 |
-
|
|
|
|
|
15 |
import torch
|
16 |
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, set_seed
|
17 |
|
@@ -39,7 +41,7 @@ for t in text:
|
|
39 |
|
40 |
## Configuration
|
41 |
|
42 |
-
| Parameters | Vocab size |
|
43 |
| :-----: | :-----------: | :-------------: | :----------- | :-----------: | :----: | :--------: | :-------------: |
|
44 |
| [7B](https://huggingface.co/sbintuitions/sarashina1-7b) | 51200 | 1.0T | GPTNeoX | RoPE | 32 | 4096 | 32 |
|
45 |
| [13B](https://huggingface.co/sbintuitions/sarashina1-13b) | 51200 | 1.0T | GPTNeoX | RoPE | 40 | 5120 | 40 |
|
|
|
11 |
|
12 |
## How to use
|
13 |
|
14 |
+
Please set **use_fast=False** to use our tokenizer properly.
|
15 |
+
|
16 |
+
```python
|
17 |
import torch
|
18 |
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, set_seed
|
19 |
|
|
|
41 |
|
42 |
## Configuration
|
43 |
|
44 |
+
| Parameters | Vocab size | Training tokens | Architecture | Position type | Layers | Hidden dim | Attention heads |
|
45 |
| :-----: | :-----------: | :-------------: | :----------- | :-----------: | :----: | :--------: | :-------------: |
|
46 |
| [7B](https://huggingface.co/sbintuitions/sarashina1-7b) | 51200 | 1.0T | GPTNeoX | RoPE | 32 | 4096 | 32 |
|
47 |
| [13B](https://huggingface.co/sbintuitions/sarashina1-13b) | 51200 | 1.0T | GPTNeoX | RoPE | 40 | 5120 | 40 |
|