<story><title>Accessing Llama 2 from the command-line with the LLM-replicate plugin</title><url>https://simonwillison.net/2023/Jul/18/accessing-llama-2/</url></story><parent_chain></parent_chain><comment><author>xrd</author><text>I&amp;#x27;m so confused about running these models locally only. When reading about the llm tool, I thought, ok, this helps organize all the pieces on my machine. But then it uses a replicate API key, so clearly it requires a network connection. Is this just to download the models? I feel like we need a new license or packaging model that clearly states whether the computation is happening locally or remotely. It&amp;#x27;s very important to me and often hard to know until I&amp;#x27;m a long way into the installation process.</text></comment>
<story><title>Accessing Llama 2 from the command-line with the LLM-replicate plugin</title><url>https://simonwillison.net/2023/Jul/18/accessing-llama-2/</url></story><parent_chain></parent_chain><comment><author>simonw</author><text>More about my LLM tool (and Python library) here: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;llm.datasette.io&amp;#x2F;&quot; rel=&quot;nofollow noreferrer&quot;&gt;https:&amp;#x2F;&amp;#x2F;llm.datasette.io&amp;#x2F;&lt;/a&gt;&lt;p&gt;Here&amp;#x27;s the full implementation of that llm-replicate plugin: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;simonw&amp;#x2F;llm-replicate&amp;#x2F;blob&amp;#x2F;0.2&amp;#x2F;llm_replicate&amp;#x2F;__init__.py&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;simonw&amp;#x2F;llm-replicate&amp;#x2F;blob&amp;#x2F;0.2&amp;#x2F;llm_replica...&lt;/a&gt;&lt;p&gt;If you want to write a plugin for some other LLM I have a detailed tutorial here: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;llm.datasette.io&amp;#x2F;en&amp;#x2F;stable&amp;#x2F;plugins&amp;#x2F;tutorial-model-plugin.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https:&amp;#x2F;&amp;#x2F;llm.datasette.io&amp;#x2F;en&amp;#x2F;stable&amp;#x2F;plugins&amp;#x2F;tutorial-model-pl...&lt;/a&gt; - plus a bunch of examples linked from here: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;simonw&amp;#x2F;llm-plugins&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;simonw&amp;#x2F;llm-plugins&lt;/a&gt;</text></comment>