<story><title>Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models</title><url>https://github.com/google/gemma.cpp</url></story><parent_chain><item><author>zoogeny</author><text>I know a lot of people chide Google for being behind OpenAI in their commercial offerings. We also dunk on them for the over-protective nature of their fine-tuning.&lt;p&gt;But Google is scarily capable on the LLM front and we shouldn&amp;#x27;t count them out. OpenAI might have the advantage of being quick to move, but when the juggernaut gets passed its resting inertia and starts to gain momentum it is going to leave an impression.&lt;p&gt;That became clear to me after watching the recent Jeff Dean video [1] which was posted a few days ago. The depth of institutional knowledge that is going to be unlocked inside Google is actually frightening for me to consider.&lt;p&gt;I hope the continued competition on the open source front, which we can really thank Facebook and Llama for, keeps these behemoths sharing. As OpenAI moves further from its original mission into capitalizing on its technological lead, we have to remember why the original vision they had is important.&lt;p&gt;So thank you, Google, for this.&lt;p&gt;1. &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=oSCRZkSQ1CE&amp;amp;ab_channel=RiceKenKennedyInstitute&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=oSCRZkSQ1CE&amp;amp;ab_channel=RiceK...&lt;/a&gt;</text></item></parent_chain><comment><author>llm_nerd</author><text>While I generally agree with you, who has ever counted Google out? We&amp;#x27;ve made fun of Google for lagging while they instead spend their engineering time renaming projects and performing algorithmic white-erasure, but we all knew they&amp;#x27;re a potent force.&lt;p&gt;Google has as much or more computing power than anyone. They&amp;#x27;re massively capitalized and have a market cap of almost $2T and colossal cashflow, and have the ability to throw enormous resources at the problem until they have a competitor. They have an enormous, benchmark-setting amount of data across their various projects to train on. That we&amp;#x27;re talking like they&amp;#x27;re some scrappy upstart is super weird.&lt;p&gt;&amp;gt;As OpenAI moves further from its original mission into capitalizing on its technological lead, we have to remember why the original vision they had is important.&lt;p&gt;I&amp;#x27;m way more cynical about the open source models released by the megas, and OpenAI is probably the most honest about their intentions. Meta and Google are releasing these models arguably to kneecap any possible next OpenAI. They want to basically set the market value of anything below state of the art at $0.00, ensuring that there is no breathing room below the $2T cos. These models (Llama, Gemma, etc) are fun toys, but in the end they&amp;#x27;re completely uncompetitive and will yield zero &amp;quot;wins&amp;quot;, so to speak.</text></comment>
<story><title>Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models</title><url>https://github.com/google/gemma.cpp</url></story><parent_chain><item><author>zoogeny</author><text>I know a lot of people chide Google for being behind OpenAI in their commercial offerings. We also dunk on them for the over-protective nature of their fine-tuning.&lt;p&gt;But Google is scarily capable on the LLM front and we shouldn&amp;#x27;t count them out. OpenAI might have the advantage of being quick to move, but when the juggernaut gets passed its resting inertia and starts to gain momentum it is going to leave an impression.&lt;p&gt;That became clear to me after watching the recent Jeff Dean video [1] which was posted a few days ago. The depth of institutional knowledge that is going to be unlocked inside Google is actually frightening for me to consider.&lt;p&gt;I hope the continued competition on the open source front, which we can really thank Facebook and Llama for, keeps these behemoths sharing. As OpenAI moves further from its original mission into capitalizing on its technological lead, we have to remember why the original vision they had is important.&lt;p&gt;So thank you, Google, for this.&lt;p&gt;1. &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=oSCRZkSQ1CE&amp;amp;ab_channel=RiceKenKennedyInstitute&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=oSCRZkSQ1CE&amp;amp;ab_channel=RiceK...&lt;/a&gt;</text></item></parent_chain><comment><author>whimsicalism</author><text>Realistically, if Google has all this talent, they should have gotten the juggernaut moving in 2020.&lt;p&gt;Google has had &lt;i&gt;years&lt;/i&gt; to get to this stage, and they&amp;#x27;ve lost a lot of the talent that made their initial big splashes to OAI and competitors. Try finding someone on a sparse MoE paper from Google prior to 2022 who is still working there and not at OAI.&lt;p&gt;With respect, they can hardly even beat Mistral, resorting to rounding down a 7.8b model (w&amp;#x2F;o embeddings) to 7b.</text></comment>