dice-research
/

lola_v1

Text Generation

Model card Files Files and versions Community

neo-nlp-dev commited on Aug 23

Commit

d7ea010

•

1 Parent(s): 8b31051

Update README.md

Files changed (1) hide show

README.md +4 -2

README.md CHANGED Viewed

@@ -11,10 +11,12 @@ tags: []
 ## Model Description
 - **Developed by:** DICE Research Group (https://dice-research.org/) @ Paderborn University (https://www.uni-paderborn.de/)
-- **Model type:** GPT2 style (decoder-only) with Mixture-of-Experts layers
 - **Language(s) (NLP):** 160+
 - **License:** Coming soon
-- **Repository:** https://github.com/dice-group/LOLA-Megatron-DeepSpeed
 ## How to Get Started with the Model

 ## Model Description
 - **Developed by:** DICE Research Group (https://dice-research.org/) @ Paderborn University (https://www.uni-paderborn.de/)
+- **Model type:** GPT2 style (decoder-only) with alternating Mixture-of-Experts layers
+- **Number of Experts**: 16
+- **Model Size**: 1.3 Billion Dense / 7.4 Billion Sparse
 - **Language(s) (NLP):** 160+
 - **License:** Coming soon
+- **Repository:** https://github.com/dice-group/LOLA
 ## How to Get Started with the Model