<story><title>LoRA: Low-Rank Adaptation of Large Language Models</title><url>https://github.com/microsoft/LoRA</url></story><parent_chain></parent_chain><comment><author>numlocked</author><text>For those wondering why this is interesting: This technique is being used to reproduce[0] the Alpaca results from Stanford[1] with a few hours of training on consumer-grade hardware.&lt;p&gt;I believe there will soon be a cottage industry of providing application-specific fine-tuned models like this, that can run in e.g. AWS very inexpensively. The barrier today seems to be that the base model (here, Meta&amp;#x27;s LLaMA) is encumbered and can&amp;#x27;t be used commercially. Someone will soon, I&amp;#x27;m confident, release e.g. an MIT-licensed equivalent and we&amp;#x27;ll all be off to the races.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;tloen&amp;#x2F;alpaca-lora&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;tloen&amp;#x2F;alpaca-lora&lt;/a&gt;&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;crfm.stanford.edu&amp;#x2F;2023&amp;#x2F;03&amp;#x2F;13&amp;#x2F;alpaca.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;crfm.stanford.edu&amp;#x2F;2023&amp;#x2F;03&amp;#x2F;13&amp;#x2F;alpaca.html&lt;/a&gt;</text></comment>
<story><title>LoRA: Low-Rank Adaptation of Large Language Models</title><url>https://github.com/microsoft/LoRA</url></story><parent_chain></parent_chain><comment><author>runnerup</author><text>There’s a not insignificant intersection of projects and developers who might be using both LoRA and LoRa at the same time. What a terrible name collision. Hopefully this doesn’t become one of the foundational terms in AI that everyone must use frequently like “Transformer”.</text></comment>