Post
1169
Mixture of Agents now in MLC/LMStudio/Ollama
I've been a bit obsessed with the recent MoA paper and its implementation. I've noticed a HUGE upgrade in the final output and it seems to really be a great way to harness the power of a team of different LLMs. The downside is that it can be a bit slow to generate responses with the bigger models (but worth it if you want to wait). I wanted to get faster results so I made an MLC version and it actually works out great! Much quicker and the responses definitely are better than compared to just running one.
I'm going to keep working on seeing how it can be further integrated (API endpoints, RAG, synthetic data generation, etc) and will share the stuff that I can get to work decently enough :)
https://github.com/severian42/MoA-MLC-Chat
https://github.com/severian42/MoA-Ollama-Chat
https://github.com/severian42/MoA-LMStudio-Chat
I've been a bit obsessed with the recent MoA paper and its implementation. I've noticed a HUGE upgrade in the final output and it seems to really be a great way to harness the power of a team of different LLMs. The downside is that it can be a bit slow to generate responses with the bigger models (but worth it if you want to wait). I wanted to get faster results so I made an MLC version and it actually works out great! Much quicker and the responses definitely are better than compared to just running one.
I'm going to keep working on seeing how it can be further integrated (API endpoints, RAG, synthetic data generation, etc) and will share the stuff that I can get to work decently enough :)
https://github.com/severian42/MoA-MLC-Chat
https://github.com/severian42/MoA-Ollama-Chat
https://github.com/severian42/MoA-LMStudio-Chat