Commit
•
716e2c1
1
Parent(s):
25cfaef
minor prose tweaks (#1)
Browse files- minor prose tweaks (555974c96b0a25d3e60593f274d61c415b56ef9c)
Co-authored-by: d blalock <[email protected]>
README.md
CHANGED
@@ -91,7 +91,7 @@ model = transformers.AutoModelForCausalLM.from_pretrained(
|
|
91 |
)
|
92 |
```
|
93 |
|
94 |
-
The model was trained
|
95 |
|
96 |
```python
|
97 |
import transformers
|
|
|
91 |
)
|
92 |
```
|
93 |
|
94 |
+
The model was trained initially on a sequence length of 2048. An additional pre-training phase was included for sequence length adaptation to 8192. However, ALiBi further enables users to increase the maximum sequence length during finetuning and/or inference. For example:
|
95 |
|
96 |
```python
|
97 |
import transformers
|