update readme

README.md

@@ -14,21 +14,29 @@ language:
 - is
 ---
 
-<div align="center">
-<img src="./viking.png" width="200px">
-</div>
 
-# Viking 33B Model Card
+# Viking 33B
 
 _**NOTE:** This is a **research checkpoint** of a model for which **training has not been completed.** It is being provided in its current state for research and testing purposes. **Care should be taken when using the outputs of the model.** Once pretraining has completed we intend to release additional instruction-tuned and chat-tuned varieties._
 
-Viking 33B is a 33B parameter decoder-only transformer pretrained on Finnish, English, Swedish, Danish, Norwegian, Icelandic and code.  It is being trained on 2 trillion tokens (700B billion as of this release). Viking 33B is a fully open source model and is made available under the Apache 2.0 License.
+Viking 33B is a 33B parameter decoder-only transformer pretrained on Finnish,
+English, Swedish, Danish, Norwegian, Icelandic and code.  It is being trained
+on 2 trillion tokens (1300B billion as of this release). Viking 33B is a fully open source model and is made available under the Apache 2.0 License.
 
-Viking was created in a collaboration between [SiloGen](https://www.silo.ai/silogen) from [Silo AI](https://www.silo.ai/), the [TurkuNLP group](https://turkunlp.org/) of the University of Turku, and [High Performance Language Technologies](https://hplt-project.org/) (HPLT). Training was conducted on the [LUMI supercomputer](https://www.lumi-supercomputer.eu/), using compute resources generously provided by [CSC](https://csc.fi/) - IT Center for Science, Finland.
+Viking was created in a collaboration between the [TurkuNLP group](https://turkunlp.org/) of the University of Turku, [SiloGen](https://www.silo.ai/silogen) from [Silo AI](https://www.silo.ai/),and [High Performance Language Technologies](https://hplt-project.org/) (HPLT). Training was conducted on the [LUMI supercomputer](https://www.lumi-supercomputer.eu/), using compute resources generously provided by [CSC](https://csc.fi/) - IT Center for Science, Finland.
 
 This project is part of an ongoing effort to create open source large language models for non-English and especially low resource languages like Finnish. The mode is fluent in Finnish, English, the Scandinavian languages and capable of basic translation between them. It is also able to understand and generate code.
 
-Viking 33B is part of the second set of models of our model family. Work is also underway on our next models which will support even more languages and grouped query attention.
+## Model Family
+
+Viking is the second set of models released by LumiOpen and is available at
+3 parameter counts:
+
+[Viking 7B](https://huggingface.co/LumiOpen/Viking-7B)
+
+[Viking 13B](https://huggingface.co/LumiOpen/Viking-13B)
+
+[Viking 33B](https://huggingface.co/LumiOpen/Viking-33B)
 
 ## Model Overview
 _**NOTE:** In addition to being an early research release, Viking is a base model which needs further fine tuning for most use cases._
@@ -44,32 +52,6 @@ Viking is a generative pretrained transformer using a LLaMA-like GPT architectur
 | vocab_size | 131072 |
 | sequence_length | 4096 |
 
-## Viking Research Checkpoints
-
-Checkpoints are available as branches in the repository.  Checkpoints will be released roughly every 100B tokens.  The main branch will always point to the latest checkpoint.  The following checkpoints are available:
-
-* [100B](https://huggingface.co/LumiOpen/Viking-33B/tree/100B)
-* [200B](https://huggingface.co/LumiOpen/Viking-33B/tree/200B)
-* [300B](https://huggingface.co/LumiOpen/Viking-33B/tree/300B)
-* [400B](https://huggingface.co/LumiOpen/Viking-33B/tree/400B)
-* [500B](https://huggingface.co/LumiOpen/Viking-33B/tree/500B)
-* [600B](https://huggingface.co/LumiOpen/Viking-33B/tree/600B)
-* [700B](https://huggingface.co/LumiOpen/Viking-33B/tree/700B)
-* [800B](https://huggingface.co/LumiOpen/Viking-33B/tree/800B)
-* [900B](https://huggingface.co/LumiOpen/Viking-33B/tree/900B)
-* [1000B](https://huggingface.co/LumiOpen/Viking-33B/tree/1000B)
-
-The transformers library allows you to load a checkpoint from a branch as follows:
-
-```python
-branch = "200B"
-model = transformers.AutoModelForCausalLM.from_pretrained(
-    "LumiOpen/Viking-33B",
-    torch_dtype=torch.bfloat16,
-    revision=branch,
-)
-```
-
 ## Training
 
 Viking 33B was trained on the LUMI supercomputer, using 1024 AMD MI250X GPUs.  Each MI250X GPU has two Graphics Complex Dies (GCDs) for a world size of 2048 during training, using activation checkpointing, a micro batch size of 1, gradient accumulation of 16, and a 3D parallelism strategy of TP=4, PP=4, DP=128.
@@ -101,6 +83,36 @@ Full details will be published soon.
 
 Full evaluation results will be published with the final model. 
 
+## Training Checkpoints
+
+Training checkpoints are available as branches in the repository.  Checkpoints will be released roughly every 100B tokens.  The main branch will always point to the latest checkpoint.  The following checkpoints are available:
+
+* [100B](https://huggingface.co/LumiOpen/Viking-33B/tree/100B)
+* [200B](https://huggingface.co/LumiOpen/Viking-33B/tree/200B)
+* [300B](https://huggingface.co/LumiOpen/Viking-33B/tree/300B)
+* [400B](https://huggingface.co/LumiOpen/Viking-33B/tree/400B)
+* [500B](https://huggingface.co/LumiOpen/Viking-33B/tree/500B)
+* [600B](https://huggingface.co/LumiOpen/Viking-33B/tree/600B)
+* [700B](https://huggingface.co/LumiOpen/Viking-33B/tree/700B)
+* [800B](https://huggingface.co/LumiOpen/Viking-33B/tree/800B)
+* [900B](https://huggingface.co/LumiOpen/Viking-33B/tree/900B)
+* [1000B](https://huggingface.co/LumiOpen/Viking-33B/tree/1000B)
+* [1100B](https://huggingface.co/LumiOpen/Viking-33B/tree/1100B)
+* [1200B](https://huggingface.co/LumiOpen/Viking-33B/tree/1200B)
+* [1300B](https://huggingface.co/LumiOpen/Viking-33B/tree/1300B)
+
+The transformers library allows you to load a checkpoint from a branch as follows:
+
+```python
+branch = "200B"
+model = transformers.AutoModelForCausalLM.from_pretrained(
+    "LumiOpen/Viking-33B",
+    torch_dtype=torch.bfloat16,
+    revision=branch,
+)
+```
+
+
 ## Ethical Considerations and Limitations
 
 _Viking 33B is a release of a partially trained model, and special care should be taken when using any output._