<story><title>Open Assistant: Conversational AI for Everyone</title><url>https://open-assistant.io/</url></story><parent_chain><item><author>naasking</author><text>&amp;gt; it is frustrating to be blocked from doing a lot of research in this space due to computational&lt;p&gt;Do we need a SETI@home-like project to distribute the 
training computation across many volunteers so we can all benefit from the trained model?</text></item><item><author>chriskanan</author><text>I&amp;#x27;m really excited about this project and I think it could be really disruptive.  It is organized by LAION, the same folks who curated the dataset used to train Stable Diffusion.&lt;p&gt;My understanding of the plan is to fine-tune an existing large language model, trained with self-supervised learning on a very large corpus of data, using reinforcement learning from human feedback, which is the same method used in ChatGPT. Once the dataset they are creating is available, though, perhaps better methods can be rapidly developed as it will democratize the ability to do basic research in this space. I&amp;#x27;m curious regarding how much more limited the systems they are planning to build will be compared to ChatGPT, since they are planning to make models with far less parameters to deploy them on much more modest hardware than ChatGPT.&lt;p&gt;As an AI researcher in academia, it is frustrating to be blocked from doing a 
lot of research in this space due to computational constraints and a lack of the required data. I&amp;#x27;m teaching a class this semester on self-supervised and generative AI methods, and it will be fun to let students play around with this in the future.&lt;p&gt;Here is a video about the Open Assistant effort: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=64Izfm24FKA&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=64Izfm24FKA&lt;/a&gt;</text></item></parent_chain><comment><author>zone411</author><text>Long story short, training requires intensive device-to-device communication. Distributed training is possible in theory but so inefficient that it&amp;#x27;s not worth it. Here is a new paper that looks to be the most promising approach yet: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;2301.11913&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;2301.11913&lt;/a&gt;</text></comment>
<story><title>Open Assistant: Conversational AI for Everyone</title><url>https://open-assistant.io/</url></story><parent_chain><item><author>naasking</author><text>&amp;gt; it is frustrating to be blocked from doing a lot of research in this space due to computational&lt;p&gt;Do we need a SETI@home-like project to distribute the 
training computation across many volunteers so we can all benefit from the trained model?</text></item><item><author>chriskanan</author><text>I&amp;#x27;m really excited about this project and I think it could be really disruptive.  It is organized by LAION, the same folks who curated the dataset used to train Stable Diffusion.&lt;p&gt;My understanding of the plan is to fine-tune an existing large language model, trained with self-supervised learning on a very large corpus of data, using reinforcement learning from human feedback, which is the same method used in ChatGPT. Once the dataset they are creating is available, though, perhaps better methods can be rapidly developed as it will democratize the ability to do basic research in this space. I&amp;#x27;m curious regarding how much more limited the systems they are planning to build will be compared to ChatGPT, since they are planning to make models with far less parameters to deploy them on much more modest hardware than ChatGPT.&lt;p&gt;As an AI researcher in academia, it is frustrating to be blocked from doing a 
lot of research in this space due to computational constraints and a lack of the required data. I&amp;#x27;m teaching a class this semester on self-supervised and generative AI methods, and it will be fun to let students play around with this in the future.&lt;p&gt;Here is a video about the Open Assistant effort: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=64Izfm24FKA&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.youtube.com&amp;#x2F;watch?v=64Izfm24FKA&lt;/a&gt;</text></item></parent_chain><comment><author>8f2ab37a-ed6c</author><text>That&amp;#x27;s brilliant, I would love to spare compute cycles and network on my devices for this if there&amp;#x27;s an open source LLM on the other side that I can use in my own projects, or commercially.&lt;p&gt;Doesn&amp;#x27;t feel like there&amp;#x27;s much competition for ChatGPT at this point otherwise, which can&amp;#x27;t be good.</text></comment>