<story><title>Understanding ChatGPT</title><url>https://www.atmosera.com/ai/understanding-chatgpt/</url></story><parent_chain><item><author>ak_111</author><text>The strongest answer to almost all of your questions is &amp;quot;Poverty of the stimulus&amp;quot; (wikipedia). 4 year olds are exposed to an almost microscopically tiny amount of words relative to chatgpt (you can probably contain it in a csv file that you can open in excel), and yet can reason, even develop multilingual skills and a huge amount of emotional intelligence from the very little word tokens they are exposed to.&lt;p&gt;So whatever is driving reasoning and intelligence in humans is clearly &lt;i&gt;very&lt;/i&gt; different to what is driving reasoning in chatgpt.&lt;p&gt;People will probably respond by saying but babies are exposed to much more data than just words, this is true, but chatgpt is learning only from words and no one has shown how you can get chatgpt to sufficiently learn what a baby learns by other kind of data. Also note that even blind babies learn language pretty quickly so this also excludes the huge amount of data you obtain from vision as putting babies at an advantage, and it is very difficult to show how sensory touch data for example contribute to babies learning to manipulate language efficiently.</text></item><item><author>maxdoop</author><text>“It’s a glorified word predictor” is becoming increasingly maddening to read.&lt;p&gt;Do tell— how can you prove humans are any different?&lt;p&gt;The most common “proofs” I’ve seen:&lt;p&gt;“Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”?&lt;p&gt;“Humans are actually reasoning. LLMs are not.” Again, how would you measure such a thing?&lt;p&gt;“LLMs are confidently wrong .” How is this relevant ? And are humans not confidently wrong as well?&lt;p&gt;“LLMs are good at single functions, but they can’t understand a system.” This is simply a matter of increasing the context limit, is it not? And was there not a leaked OpenAI document showing a future offering of 64k tokens?&lt;p&gt;All that aside, I’m forever amazed how a seemingly forward-looking group of people is continually dismissive of a tool that came out LITERALLY 4 MONTHS AGO, with its latest iteration less than TWO WEEKS ago. For people familiar with stuff like Moore’s law, it’s absolutely wild to see how people act like LLM progress is forever tied to its current , apparently static, state.</text></item></parent_chain><comment><author>NaN1352</author><text>I think what’s fascinating about GPT developments is it may very well emulate part of our left brain hemisphere. As McGilchrist pointed out, with lots of research and studies on people with brain damage strokes etc. is that it is the left hemisphere that sees the world from an internal representation. Right hemi. sees a car as the &amp;quot;thing in itself&amp;quot; in direct experience (colours etc). The left hemisphere makes it into a category of &amp;quot;car&amp;quot;, now removed from direct experience.&lt;p&gt;And just like gpt’s data set isnt necessarily truth, so is our own image of the world which as we know can be deeply distorted through abusive childhood, cults etc. In fact, all of human knowledge is simply beliefs, agreed stories about reality. For example &amp;quot;red&amp;quot; is a word&amp;#x2F;sound that points to an experience. The word alone only has meaning in context (what GPT can handle), but can never substitue for a conscious experience.&lt;p&gt;Crucially imho, software will never be able to do what the right hemisphere does. And I find it dumbfounding that even Lex Fridman doesnt see the fundamental difference between conceptual thought &amp;#x2F; language based reasoning, and direct experience aka consciousness.</text></comment>
<story><title>Understanding ChatGPT</title><url>https://www.atmosera.com/ai/understanding-chatgpt/</url></story><parent_chain><item><author>ak_111</author><text>The strongest answer to almost all of your questions is &amp;quot;Poverty of the stimulus&amp;quot; (wikipedia). 4 year olds are exposed to an almost microscopically tiny amount of words relative to chatgpt (you can probably contain it in a csv file that you can open in excel), and yet can reason, even develop multilingual skills and a huge amount of emotional intelligence from the very little word tokens they are exposed to.&lt;p&gt;So whatever is driving reasoning and intelligence in humans is clearly &lt;i&gt;very&lt;/i&gt; different to what is driving reasoning in chatgpt.&lt;p&gt;People will probably respond by saying but babies are exposed to much more data than just words, this is true, but chatgpt is learning only from words and no one has shown how you can get chatgpt to sufficiently learn what a baby learns by other kind of data. Also note that even blind babies learn language pretty quickly so this also excludes the huge amount of data you obtain from vision as putting babies at an advantage, and it is very difficult to show how sensory touch data for example contribute to babies learning to manipulate language efficiently.</text></item><item><author>maxdoop</author><text>“It’s a glorified word predictor” is becoming increasingly maddening to read.&lt;p&gt;Do tell— how can you prove humans are any different?&lt;p&gt;The most common “proofs” I’ve seen:&lt;p&gt;“Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”?&lt;p&gt;“Humans are actually reasoning. LLMs are not.” Again, how would you measure such a thing?&lt;p&gt;“LLMs are confidently wrong .” How is this relevant ? And are humans not confidently wrong as well?&lt;p&gt;“LLMs are good at single functions, but they can’t understand a system.” This is simply a matter of increasing the context limit, is it not? And was there not a leaked OpenAI document showing a future offering of 64k tokens?&lt;p&gt;All that aside, I’m forever amazed how a seemingly forward-looking group of people is continually dismissive of a tool that came out LITERALLY 4 MONTHS AGO, with its latest iteration less than TWO WEEKS ago. For people familiar with stuff like Moore’s law, it’s absolutely wild to see how people act like LLM progress is forever tied to its current , apparently static, state.</text></item></parent_chain><comment><author>procgen</author><text>There&amp;#x27;s billions of years of compressed knowledge in those 4 year olds. Lots of useful priors.</text></comment>