<story><title>AI’s Language Problem</title><url>https://www.technologyreview.com/s/602094/ais-language-problem/?set=602129</url></story><parent_chain><item><author>erikpukinskis</author><text>No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don&amp;#x27;t understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I&amp;#x27;m not saying they have to be exactly like us, but they will only be able to understand us to the extent that they have body parts and social experiences that are analogous to ours.&lt;p&gt;This isn&amp;#x27;t an engineering problem, it&amp;#x27;s a philosophical problem: We are blind to most things. We can only see what we can relate to personally. We can use language and other symbol systems to expand our understanding by permuting and recombining our personal experiences, but everything is grounded in our interactive developmental trajectory.&lt;p&gt;The kitten-in-a-cart experiment demonstrates this clearly: &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;io9.gizmodo.com&amp;#x2F;the-seriously-creepy-two-kitten-experiment-1442107174&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;io9.gizmodo.com&amp;#x2F;the-seriously-creepy-two-kitten-exper...&lt;/a&gt;  Interaction is crucial for perception. Sensation is not experience.&lt;p&gt;And here&amp;#x27;s the rub: Once you give an AI an animal-like personal developmental trajectory to use for grounding the semantics of their symbol systems, you end up with something which is not particularly different or better than a human cyborg.</text></item></parent_chain><comment><author>Houshalter</author><text>I believe we can get AI from just text. Obviously that won&amp;#x27;t work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours!&lt;p&gt;Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vector &amp;quot;king&amp;quot; minus the vector for &amp;quot;man&amp;quot; plus &amp;quot;woman&amp;quot; equals &amp;quot;queen&amp;quot;. It&amp;#x27;s learning the meaning of words, and the relationships between them.&lt;p&gt;Recurrent NNs, using similar techniques, but with much more complexity, can learn to predict the next word in a sentence very well. They can learn to write responses that are almost indistinguishable from humans. And it&amp;#x27;s incredible this works at all, given RNNs have only a few thousand neurons at most, and a few days of training, compared to humans&amp;#x27; billions of neurons trained over a lifetime.&lt;p&gt;All of the information of our world is contained in text. Humans have produced billions of books, papers, articles, and internet comments. Billions of times more information than any human could read in their entire lifetime. Any information you can imagine is contained in text somewhere. I don&amp;#x27;t think it&amp;#x27;s necessary for AIs to be able to see, or interact with the world in any way.&lt;p&gt;If you can predict the word a human would say next, with enough accuracy, then you could also produce answers indistinguishable from theirs. Meaning you could pass the Turing test, and perform any language task they could do just as well. So language prediction alone may be sufficient for AGI.&lt;p&gt;This is the theory behind the Hutter Prize, which proposes that predicting (compressing) wikipedia&amp;#x27;s text is a measure of AI progress. The Hutter Prize isn&amp;#x27;t perfect (it&amp;#x27;s only a sample of wikipedia, which is very small compared to all the text humans have produced), but the idea is solid.</text></comment>
<story><title>AI’s Language Problem</title><url>https://www.technologyreview.com/s/602094/ais-language-problem/?set=602129</url></story><parent_chain><item><author>erikpukinskis</author><text>No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don&amp;#x27;t understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I&amp;#x27;m not saying they have to be exactly like us, but they will only be able to understand us to the extent that they have body parts and social experiences that are analogous to ours.&lt;p&gt;This isn&amp;#x27;t an engineering problem, it&amp;#x27;s a philosophical problem: We are blind to most things. We can only see what we can relate to personally. We can use language and other symbol systems to expand our understanding by permuting and recombining our personal experiences, but everything is grounded in our interactive developmental trajectory.&lt;p&gt;The kitten-in-a-cart experiment demonstrates this clearly: &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;io9.gizmodo.com&amp;#x2F;the-seriously-creepy-two-kitten-experiment-1442107174&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;io9.gizmodo.com&amp;#x2F;the-seriously-creepy-two-kitten-exper...&lt;/a&gt;  Interaction is crucial for perception. Sensation is not experience.&lt;p&gt;And here&amp;#x27;s the rub: Once you give an AI an animal-like personal developmental trajectory to use for grounding the semantics of their symbol systems, you end up with something which is not particularly different or better than a human cyborg.</text></item></parent_chain><comment><author>digitalarborist</author><text>It&amp;#x27;s probably a mistake to assume that just because it&amp;#x27;s the way we do it that it has to be the way machines do it. Although that&amp;#x27;s usually the initial assumption. In the early days of flight most attempts were based on birds, similarly submersible vehicles were based on fish. We know now it&amp;#x27;s better to use propellers. It could be we just haven&amp;#x27;t found what is analogous to a propeller for the AI world.</text></comment>