<story><title>Releasing 3B and 7B RedPajama</title><url>https://www.together.xyz/blog/redpajama-models-v1</url></story><parent_chain></parent_chain><comment><author>sphars</author><text>Slightly off-topic, but as the parent of a toddler, I got a bit of a chuckle out of the name. It&amp;#x27;s based off the children&amp;#x27;s book series of &amp;quot;Llama Llama Red Pajama&amp;quot;</text></comment>
<story><title>Releasing 3B and 7B RedPajama</title><url>https://www.together.xyz/blog/redpajama-models-v1</url></story><parent_chain></parent_chain><comment><author>andy_xor_andrew</author><text>This is beyond exciting. Welcome to the new reality!&lt;p&gt;On one hand, the resources required to run these models continues falling dramatically, thanks to the techniques discovered by researchers: GPTQ quantizing down to 4, 3, 2, even 1 bits! model pruning! hybrid vram offloading! better, more efficient architectures! 1-click finetuning on consumer hardware!  Of course, the free lunches won&amp;#x27;t last forever, and this will level off, but it&amp;#x27;s still incredible.&lt;p&gt;And on the other side of the coin, the power of &lt;i&gt;all&lt;/i&gt; computing devices continues its ever-upward exponential growth.&lt;p&gt;So you have a continuous &lt;i&gt;lowering&lt;/i&gt; of requirements, combined with a continuous &lt;i&gt;increase&lt;/i&gt; in available power... surely these two trends will collide, and I can only imagine what this stuff will be like at that intersection.</text></comment>