<story><title>Running a 180B parameter LLM on a single Apple M2 Ultra</title><url>https://twitter.com/ggerganov/status/1699791226780975439</url></story><parent_chain></parent_chain><comment><author>adam_arthur</author><text>Even a linear growth rate of average RAM capacity would obviate the need to run current SOTA LLMs remotely in short order.&lt;p&gt;Historically average RAM has grown far faster than linear, and there really hasn&amp;#x27;t been anything pressing manufacturers to push the envelope here in the past few years... until now.&lt;p&gt;It could be that LLM model sizes keep increasing such that we continue to require cloud consumption, but I suspect the sizes will not increase as quickly as hardware for inference.&lt;p&gt;Given how useful GPT-4 is already. Maybe one more iteration would unlock the vast majority of practical use cases.&lt;p&gt;I think people will be surprised  that consumers ultimately end up benefitting far more from LLMs than the providers. There&amp;#x27;s not going to be much moat or differentiation to defend margins... more of a race to the bottom on pricing</text></comment>
<story><title>Running a 180B parameter LLM on a single Apple M2 Ultra</title><url>https://twitter.com/ggerganov/status/1699791226780975439</url></story><parent_chain></parent_chain><comment><author>logicchains</author><text>Pretty amazing that in such a short span of time we went from people being amazed how powerful GPT3.5 was upon its release to people being able to run something equivalently powerful locally.</text></comment>