<story><title>How will AI learn next?</title><url>https://www.newyorker.com/science/annals-of-artificial-intelligence/how-will-ai-learn-next</url></story><parent_chain></parent_chain><comment><author>rented_mule</author><text>Anyone who has iterated on trained models for long enough knows that feedback loops can be a serious problem. If your models are influencing the generation of data that they are later retrained on, it gets harder and harder to even maintain model performance. The article mentions one experiment in this direction: &amp;quot;With each generation, the quality of the model actually degraded.&amp;quot; This happens whenever there aren&amp;#x27;t solid strategies to avoid feedback loop issues.&lt;p&gt;Given this, the problem isn&amp;#x27;t just that there&amp;#x27;s not enough new content. It&amp;#x27;s that an ever-increasing fraction of the content in the public sphere will be generated by these models. And can the models detect that they are ingesting their own output? If they get good enough, they probably can&amp;#x27;t. And then they&amp;#x27;ll get worse.&lt;p&gt;This could have a strange impact on human language &amp;#x2F; communication as well. As these models are increasingly trained on their own output, they&amp;#x27;ll start emulating their own mistakes and more of the content we consume will have these mistakes consistently used. You can imagine people, sometimes intentionally and sometimes not, starting to emulate these patterns and causing shifts in human languages. Interesting times ahead...</text></comment>
<story><title>How will AI learn next?</title><url>https://www.newyorker.com/science/annals-of-artificial-intelligence/how-will-ai-learn-next</url></story><parent_chain></parent_chain><comment><author>robbrown451</author><text>AlphaZero demonstrates that more human-generated data isn&amp;#x27;t the only thing that makes an AI smarter. It uses zero human data to learn to play Go, and just iterates. As long as it has a way of scoring itself objectively (which it obviously does with a game like Go), it can keep improving with literally no ceiling to how much it can improve.&lt;p&gt;Pretty soon ChatGPT will be able to do a lot of training by iterating on its own output, such as by writing code and analyzing the output (including using vision systems).&lt;p&gt;Here&amp;#x27;s an interesting thing I noticed last night. I have been making a lot of images that have piano keyboards in them. DALL-E 3 makes some excellent images otherwise (faces and hands mostly look great), but it always messes up the keyboards, as it doesn&amp;#x27;t seem to get how black keys are in alternating groups of two and three.&lt;p&gt;But I tried getting chatgpt to analyze an image, using its new &amp;quot;vision&amp;quot; capabilities, and the first thing it noticed was that the piano keys were not properly clustered. I said nothing about that, I just asked it &amp;quot;what is wrong with this image&amp;quot; and it immediately found that.  What if it could feed this sort of thing back in, using similar logic to Alpha Zero?&lt;p&gt;That&amp;#x27;s just a tiny hint of what is to come. Sure, it typically needs human generated data for most things. It&amp;#x27;s already got thousands of times more than any human has looked at. It will also be able to learn from human feedback, for instance a human could tell it what it got wrong in a response (whether regular text, code, or image), and explain in natural language where it deviated from what was expected. It can learn which humans are reliable, so it can minimize the number of paid employees doing RLHF, using them mostly to rate (unpaid) humans who choose to provide feedback. Even if most users opt out of giving this sort of feedback, there will be plenty to give it new, good information.</text></comment>