<story><title>We&apos;re afraid language models aren&apos;t modeling ambiguity</title><url>https://arxiv.org/abs/2304.14399</url></story><parent_chain><item><author>tinsmith</author><text>I mean, humans do that. We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we&amp;#x27;ll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.</text></item><item><author>andrewmcwatters</author><text>Not only is it possible that LLMs fail to differentiate ambiguity, but OpenAI’s flavor of GPTs fail other language understanding mechanisms as well and it’s jarring.&lt;p&gt;You can get it to mistake « afraid » between fear and sorry-to-say scenarios but you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway.&lt;p&gt;So which is it? It’s clear transformers can’t understand either case. They’re not architecturally designed to. The emergent behavior of appearing to do so is only driven by how much data you throw at them.</text></item></parent_chain><comment><author>quickthrower2</author><text>The reason humans do this and LLMs is very different. Humans do it as a social skill &amp;#x2F; tribe fitting behaviour. Agreeableness. (watch out for that!)</text></comment>
<story><title>We&apos;re afraid language models aren&apos;t modeling ambiguity</title><url>https://arxiv.org/abs/2304.14399</url></story><parent_chain><item><author>tinsmith</author><text>I mean, humans do that. We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we&amp;#x27;ll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.</text></item><item><author>andrewmcwatters</author><text>Not only is it possible that LLMs fail to differentiate ambiguity, but OpenAI’s flavor of GPTs fail other language understanding mechanisms as well and it’s jarring.&lt;p&gt;You can get it to mistake « afraid » between fear and sorry-to-say scenarios but you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway.&lt;p&gt;So which is it? It’s clear transformers can’t understand either case. They’re not architecturally designed to. The emergent behavior of appearing to do so is only driven by how much data you throw at them.</text></item></parent_chain><comment><author>civilized</author><text>Humans can be consistent if we try. LLMs can&amp;#x27;t, even when prompted to be consistent, because they don&amp;#x27;t really understand what it means to be consistent.</text></comment>