Meow. This an experimental mixture of expert model with just 2 experts based on Llama 3 Instruct plain in combo with finetune. Specifically, it is built on top of the Meta-Llama-3-8B-Instruct model and finetune is trained on Argilla Capybara dataset.
Experimental mixture of 2 experts Llama3-8b-Instruct
Built with Llama 3
- Downloads last month
- 11
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for nisten/llama3-2x8b-MoE-41k-experiment1
Base model
meta-llama/Meta-Llama-3-8B-Instruct