<story><title>Qwen1.5-Moe: Matching 7B Model Performance with 1/3 Activated Parameters</title><url>https://qwenlm.github.io/blog/qwen-moe/</url></story><parent_chain></parent_chain><comment><author>kristianp</author><text>It&amp;#x27;s actually a 14.3B parameter model.  It&amp;#x27;s irritating that they don&amp;#x27;t follow the convention of naming the model according to size.  Qwen1.5-MoE-A2.7B is named for the 2.7B activated parameters.  I guess it helps to obfuscate the size, given that it performs about as well as Mistral 7B.</text></comment>
<story><title>Qwen1.5-Moe: Matching 7B Model Performance with 1/3 Activated Parameters</title><url>https://qwenlm.github.io/blog/qwen-moe/</url></story><parent_chain></parent_chain><comment><author>cchance</author><text>Silly question, but is it possible for something similar to MoE for diffusion models like Stable Diffusion &amp;#x2F; DallE</text></comment>