jonabur
commited on
Commit
•
b9b3e0f
1
Parent(s):
8f2c8dd
update readme
Browse files
README.md
CHANGED
@@ -16,8 +16,6 @@ language:
|
|
16 |
|
17 |
# Viking 13B
|
18 |
|
19 |
-
_**NOTE:** This is a **research checkpoint** of a model for which **training has not been completed.** It is being provided in its current state for research and testing purposes. **Care should be taken when using the outputs of the model.** Once pretraining has completed we intend to release additional instruction-tuned and chat-tuned varieties._
|
20 |
-
|
21 |
Viking 13B is a 13B parameter decoder-only transformer pretrained on Finnish,
|
22 |
English, Swedish, Danish, Norwegian, Icelandic and code. It is being trained
|
23 |
on 2 trillion tokens (1.3 trillion as of this release). Viking 13B is a fully open source model and is made available under the Apache 2.0 License.
|
@@ -38,7 +36,7 @@ Viking is the second set of models released by LumiOpen and is available at
|
|
38 |
[Viking 33B](https://huggingface.co/LumiOpen/Viking-33B)
|
39 |
|
40 |
## Model Overview
|
41 |
-
_**NOTE:**
|
42 |
|
43 |
Viking is a generative pretrained transformer using a LLaMA-like GPT architecture, and makes use of rotary positional embeddings and flash attention.
|
44 |
|
@@ -102,6 +100,9 @@ Training checkpoints are available as branches in the repository. Checkpoints w
|
|
102 |
* [1500B](https://huggingface.co/LumiOpen/Viking-13B/tree/1500B)
|
103 |
* [1600B](https://huggingface.co/LumiOpen/Viking-13B/tree/1600B)
|
104 |
* [1700B](https://huggingface.co/LumiOpen/Viking-13B/tree/1700B)
|
|
|
|
|
|
|
105 |
|
106 |
The transformers library allows you to load a checkpoint from a branch as follows:
|
107 |
|
|
|
16 |
|
17 |
# Viking 13B
|
18 |
|
|
|
|
|
19 |
Viking 13B is a 13B parameter decoder-only transformer pretrained on Finnish,
|
20 |
English, Swedish, Danish, Norwegian, Icelandic and code. It is being trained
|
21 |
on 2 trillion tokens (1.3 trillion as of this release). Viking 13B is a fully open source model and is made available under the Apache 2.0 License.
|
|
|
36 |
[Viking 33B](https://huggingface.co/LumiOpen/Viking-33B)
|
37 |
|
38 |
## Model Overview
|
39 |
+
_**NOTE:** This is a base model which needs further fine tuning for most use cases._
|
40 |
|
41 |
Viking is a generative pretrained transformer using a LLaMA-like GPT architecture, and makes use of rotary positional embeddings and flash attention.
|
42 |
|
|
|
100 |
* [1500B](https://huggingface.co/LumiOpen/Viking-13B/tree/1500B)
|
101 |
* [1600B](https://huggingface.co/LumiOpen/Viking-13B/tree/1600B)
|
102 |
* [1700B](https://huggingface.co/LumiOpen/Viking-13B/tree/1700B)
|
103 |
+
* [1800B](https://huggingface.co/LumiOpen/Viking-13B/tree/1800B)
|
104 |
+
* [1900B](https://huggingface.co/LumiOpen/Viking-13B/tree/1900B)
|
105 |
+
* [2000B](https://huggingface.co/LumiOpen/Viking-13B/tree/2000B)
|
106 |
|
107 |
The transformers library allows you to load a checkpoint from a branch as follows:
|
108 |
|