Edit model card

This is a tiny Jamba model used for development, debugging and experimentation over the Jamba architecture.

It has 319M parameters (instead of 52B in Jamba 1.5 Mini (and Jamba v0.1) and 398B in Jamba 1.5 Large), and was trained on ~40B tokens.

It is great for use in unittests since it is a small model (doesn't take long to download) that has valid and non-random outputs. Yet, it did not undergo extensive training and should not be expected to generate high-quality text.

Downloads last month
9,159
Safetensors
Model size
319M params
Tensor type
BF16
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for ai21labs/Jamba-tiny-dev

Finetunes
2 models