Update SuperNova-Medius with a merge with Qwen/Qwen2.5-Coder-14B-Instruct + Further Training 😋
Qwen just dropped their 14B coder instruct model. Arcee-AI, it would be amazing if you merged SuperNova-Medius with Qwen/Qwen2.5-Coder-14B-Instruct and further trained it. 😋
Am, I dreaming, or is this viable? Qwen/Qwen2.5-Coder-14B-Instruct >= GPT4 at coding. 😋
Not a bad idea. We have quite a few models in the works right now, but I'll make sure this gets on our radar.
There's a need for a better-trained coder model! Fill-in-the-middle is especially important. Is it just me, or does nothing so far do autocomplete on local Continue.dev as well as Deepseek V2 Lite? I'd like to see that change.
@Crystalcareai Thanks! Your guys' models are helping me with my University studies. It's amazing how much smarter SMOL LLMs become when you use the logits from a Large Foundation Model to distill knowledge and capabilities into them. I can't wait for what you guys are cooking SMOL-wise. Is it possible to get a list of base models, which are being used, please? 🙏
Qwen just dropped their 14B coder instruct model. Arcee-AI, it would be amazing if you merged SuperNova-Medius with Qwen/Qwen2.5-Coder-14B-Instruct and further trained it. 😋
Am, I dreaming, or is this viable? Qwen/Qwen2.5-Coder-14B-Instruct >= GPT4 at coding. 😋
you are not dreaming! 😋
https://huggingface.co/arcee-ai/SuperNova-Medius/discussions/4#6717ebeb276e0b54768d19c8
Hey, this is such an interesting idea. Do we have any update when this will come under processing?
@MaziyarPanahi
@Crystalcareai
Thanks
We’re currently training around 10 different models—not to mention the ~15 our Customer Success team is working on for clients—so there are about 25 models in progress across Arcee right now. Some are aimed for public release, while others are part of closed-source offerings slated to be available around December 2nd. A strong 14B coder model is definitely in development, though the release date is still to be determined. We’re firing on all cylinders and will have some exciting updates to share soon!
And some of those updates won’t be just models! 😉
I’m excited! 🤩 By the way, does Arcee-AI have a discord all of us fans/enthusiasts could join? Huggingface is great, but the flow of information is not as optimal as it could be. If you guys have a discord, please link it. I would be most appreciative of it. 😋
we've thought about doing a mergekit discord - but we don't have anything at this moment. I'll renew the discussions!