<story><title>LLMs confabulate not hallucinate</title><url>https://www.beren.io/2023-03-19-LLMs-confabulate-not-hallucinate/</url></story><parent_chain><item><author>galaxyLogic</author><text>I had the same thought that &amp;quot;hallucinate&amp;quot; is too strong a term for what LLMs are doing.&lt;p&gt;Instead I would say they DREAM. Dreams follow some logic but basically they are disjoint from reality. They involve same characters as our real-life experiences but what those characters do in a dream is not based on reality.&lt;p&gt;So one could think that ALL LLMs ever do is dream but much of the time their dreams are dreams which feel very real.&lt;p&gt;Our dreams are basically an LLM based on our real experiences but recombined in arbitrary ways, still retaining some logical structure but now detached from reality because when you recombine different experiences including  our thoughts from real life, the result cannot really reflect reality very well.  LLMs are better in this respect than our dreams, but sometimes their &amp;quot;dreams&amp;quot; get really obviously detached from reality.&lt;p&gt;Or if you want to use the term &amp;quot;hallucinate&amp;quot; with LLMs then fine but its more like  they hallucinate all the time, it&amp;#x27;s just that sometimes, even often, their hallucinations seem to agree with the reality so well that we cannot tell the difference.&lt;p&gt;LLMs do not describe reality because they cannot experience reality, all they can give us is an &amp;quot;average description&amp;quot; created from many existing descriptions.&lt;p&gt;One way of looking at it is to say that all LLMs tell us is hearsay.</text></item><item><author>lachlan_gray</author><text>This reminds me of something from “The language instinct” by Steven pinker. One thing that stuck with me was a story of a girl named Denys who is or was severely mentally handicapped, but speaks perfectly.&lt;p&gt;She can’t read, write, handle money, but confabulates engaging and convincing stories about receiving mail, haggling with her bank, her boyfriend. She speaks with proper grammar, intonation, etc, but doesn’t demonstrate any understanding of what she talks about beyond the words. She doesn’t have a bank account or a boyfriend.&lt;p&gt;Trying to find more and I can’t find anything about her on the internet, I would have to look at the references. Maybe it’s questionable.&lt;p&gt;Anyway, I like to think about what the hell we even mean when we say we “know something”, and where we should actually put the bar for AI. Where is the line between confabulation and knowledge? Is knowledge just confabulation that happens to be correct?&lt;p&gt;I feel like it’s possible that Denys could feel that she “knows” what she is saying, but is simply trapped in, with a working knowledge of the world and is just unable to translate it into actions, and words are just her only degree of freedom.&lt;p&gt;I don’t remember where I was going with this, but I agree that “confabulation” is a much better term to use, but we also shouldn’t discount what it means to be able to confabulate.</text></item></parent_chain><comment><author>KaiserPro</author><text>Dreams involve the concept of self, consciousness and an understanding of reality. all of it is far too anthropomorphic.&lt;p&gt;Hallucinations are much better described as noise. Its noise that lowers the quality of the signal. its just unlike white noise, this is in the form of coherent sentences&amp;#x2F;imagery.</text></comment>
<story><title>LLMs confabulate not hallucinate</title><url>https://www.beren.io/2023-03-19-LLMs-confabulate-not-hallucinate/</url></story><parent_chain><item><author>galaxyLogic</author><text>I had the same thought that &amp;quot;hallucinate&amp;quot; is too strong a term for what LLMs are doing.&lt;p&gt;Instead I would say they DREAM. Dreams follow some logic but basically they are disjoint from reality. They involve same characters as our real-life experiences but what those characters do in a dream is not based on reality.&lt;p&gt;So one could think that ALL LLMs ever do is dream but much of the time their dreams are dreams which feel very real.&lt;p&gt;Our dreams are basically an LLM based on our real experiences but recombined in arbitrary ways, still retaining some logical structure but now detached from reality because when you recombine different experiences including  our thoughts from real life, the result cannot really reflect reality very well.  LLMs are better in this respect than our dreams, but sometimes their &amp;quot;dreams&amp;quot; get really obviously detached from reality.&lt;p&gt;Or if you want to use the term &amp;quot;hallucinate&amp;quot; with LLMs then fine but its more like  they hallucinate all the time, it&amp;#x27;s just that sometimes, even often, their hallucinations seem to agree with the reality so well that we cannot tell the difference.&lt;p&gt;LLMs do not describe reality because they cannot experience reality, all they can give us is an &amp;quot;average description&amp;quot; created from many existing descriptions.&lt;p&gt;One way of looking at it is to say that all LLMs tell us is hearsay.</text></item><item><author>lachlan_gray</author><text>This reminds me of something from “The language instinct” by Steven pinker. One thing that stuck with me was a story of a girl named Denys who is or was severely mentally handicapped, but speaks perfectly.&lt;p&gt;She can’t read, write, handle money, but confabulates engaging and convincing stories about receiving mail, haggling with her bank, her boyfriend. She speaks with proper grammar, intonation, etc, but doesn’t demonstrate any understanding of what she talks about beyond the words. She doesn’t have a bank account or a boyfriend.&lt;p&gt;Trying to find more and I can’t find anything about her on the internet, I would have to look at the references. Maybe it’s questionable.&lt;p&gt;Anyway, I like to think about what the hell we even mean when we say we “know something”, and where we should actually put the bar for AI. Where is the line between confabulation and knowledge? Is knowledge just confabulation that happens to be correct?&lt;p&gt;I feel like it’s possible that Denys could feel that she “knows” what she is saying, but is simply trapped in, with a working knowledge of the world and is just unable to translate it into actions, and words are just her only degree of freedom.&lt;p&gt;I don’t remember where I was going with this, but I agree that “confabulation” is a much better term to use, but we also shouldn’t discount what it means to be able to confabulate.</text></item></parent_chain><comment><author>orobinson</author><text>But do LLMs dream of electric sheep?</text></comment>