metadata
pipeline_tag: text-generation
language:
- en
This is a pure sub-quadtratic linear attention 8B parameter model, linearized from the Meta Llama 3.1 8B model.
Details on this model and how to train your own are provided at: https://github.com/HazyResearch/lolcats/tree/lolcats-scaled
Demo
Here is a quick GitHub GIST that will help you run inference on the model checkpoints.
Paper
See the paper page: https://huggingface.co/papers/2410.10254