<story><title>Bard is getting better at logic and reasoning</title><url>https://blog.google/technology/ai/bard-improved-reasoning-google-sheets-export/</url></story><parent_chain><item><author>drones</author><text>&amp;gt; I’m not sure these types of prompt tricks are a good way of measuring logic&lt;p&gt;They are, you just have to be creative with it. And what they demonstrate is that all of these LLM&amp;#x27;s can&amp;#x27;t reason, they only know how to parrot back what they think you want.&lt;p&gt;&amp;quot;What’s heavier, a kilogram of steel or two kilograms of one kilogram feathers?&amp;quot;&lt;p&gt;GPT: A kilogram of steel is heavier than two kilograms of feathers.&lt;p&gt;&amp;quot;Why is a kilogram of steel heavier than two kilograms of feathers?&amp;quot;&lt;p&gt;GPT: This is because steel is a much denser material than feathers. Steel is made up of atoms that are much closer together than the atoms that make up feathers, making it heavier for its size.&lt;p&gt;Edit: This was with GPT 3.5</text></item><item><author>devjab</author><text>GPT seems to get improvements of trap questions when they reach social popularity. Even the free version of ChatGPT now knows that a kilogram of feathers weighs the same as a kilogram of lead, and it didn’t always know that.&lt;p&gt;I’m not sure these types of prompt tricks are a good way of measuring logic unless Google is also implementing these directly into Bard when the hilarious outputs reach enough traction on social media.&lt;p&gt;I do wonder how OpenAI fix these logical blunders.&lt;p&gt;My biggest issue with both isn’t that they fall into these traps though. It’s that I can get them to tell me long stories about what happens in Horus Heresy books that never actually happened.  Whether the info comes from questionable sources or they are just making things up is sort of irrelevant to me, what “scares” me about those conversations is how true the answers sound, and if they are “lying” about the Horus Heresy then what else will they lie about? Don’t get me wrong, GPT now writes virtually all my JSDoc documentation and it continues to impress me when doing so, but I’m very reluctant to use it for actual information. Not only because of my time wasting conversations about the Horus Heresy but also because we’ve had it “invent” C# functions that had never existed in any version of .Net or C# when tasked to solve problems. I just mention the HH as an example because it’s fun to ask GPT why Magnus did nothing&amp;#x2F;everything wrong during meetings.</text></item><item><author>underyx</author><text>Trying my favorite LLM prompt to benchmark reasoning, as I mentioned in a thread four weeks ago[0].&lt;p&gt;&amp;gt; I&amp;#x27;m playing assetto corsa competizione, and I need you to tell me how many liters of fuel to take in a race. The qualifying time was 2:04.317, the race is 20 minutes long, and the car uses 2.73 liters per lap.&lt;p&gt;The correct answer is around 29, which GPT-4 has always known, but Bard just gave me 163.8, 21, and 24.82 as answers across three drafts.&lt;p&gt;What&amp;#x27;s even weirder is that Bard&amp;#x27;s first draft output ten lines of (wrong) Python code to calculate the result, even though my prompt mentioned nothing coding related. I wonder how non-technical users will react to this behavior. Another interesting thing is that the code follows Google&amp;#x27;s style guides.&lt;p&gt;[0]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=35893130&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=35893130&lt;/a&gt;</text></item></parent_chain><comment><author>atxbcp</author><text>Just tried the first prompt with ChatGPT... : &amp;quot;One kilogram of steel and two kilograms of feathers weigh the same. The weight of an object is determined by its mass, not the material it is made of. In this case, one kilogram is equal to two kilograms, so they have the same weight. However, it&amp;#x27;s important to note that the volume or size of the objects may be different due to the difference in density between steel and feathers.&amp;quot;
Okay...</text></comment>
<story><title>Bard is getting better at logic and reasoning</title><url>https://blog.google/technology/ai/bard-improved-reasoning-google-sheets-export/</url></story><parent_chain><item><author>drones</author><text>&amp;gt; I’m not sure these types of prompt tricks are a good way of measuring logic&lt;p&gt;They are, you just have to be creative with it. And what they demonstrate is that all of these LLM&amp;#x27;s can&amp;#x27;t reason, they only know how to parrot back what they think you want.&lt;p&gt;&amp;quot;What’s heavier, a kilogram of steel or two kilograms of one kilogram feathers?&amp;quot;&lt;p&gt;GPT: A kilogram of steel is heavier than two kilograms of feathers.&lt;p&gt;&amp;quot;Why is a kilogram of steel heavier than two kilograms of feathers?&amp;quot;&lt;p&gt;GPT: This is because steel is a much denser material than feathers. Steel is made up of atoms that are much closer together than the atoms that make up feathers, making it heavier for its size.&lt;p&gt;Edit: This was with GPT 3.5</text></item><item><author>devjab</author><text>GPT seems to get improvements of trap questions when they reach social popularity. Even the free version of ChatGPT now knows that a kilogram of feathers weighs the same as a kilogram of lead, and it didn’t always know that.&lt;p&gt;I’m not sure these types of prompt tricks are a good way of measuring logic unless Google is also implementing these directly into Bard when the hilarious outputs reach enough traction on social media.&lt;p&gt;I do wonder how OpenAI fix these logical blunders.&lt;p&gt;My biggest issue with both isn’t that they fall into these traps though. It’s that I can get them to tell me long stories about what happens in Horus Heresy books that never actually happened.  Whether the info comes from questionable sources or they are just making things up is sort of irrelevant to me, what “scares” me about those conversations is how true the answers sound, and if they are “lying” about the Horus Heresy then what else will they lie about? Don’t get me wrong, GPT now writes virtually all my JSDoc documentation and it continues to impress me when doing so, but I’m very reluctant to use it for actual information. Not only because of my time wasting conversations about the Horus Heresy but also because we’ve had it “invent” C# functions that had never existed in any version of .Net or C# when tasked to solve problems. I just mention the HH as an example because it’s fun to ask GPT why Magnus did nothing&amp;#x2F;everything wrong during meetings.</text></item><item><author>underyx</author><text>Trying my favorite LLM prompt to benchmark reasoning, as I mentioned in a thread four weeks ago[0].&lt;p&gt;&amp;gt; I&amp;#x27;m playing assetto corsa competizione, and I need you to tell me how many liters of fuel to take in a race. The qualifying time was 2:04.317, the race is 20 minutes long, and the car uses 2.73 liters per lap.&lt;p&gt;The correct answer is around 29, which GPT-4 has always known, but Bard just gave me 163.8, 21, and 24.82 as answers across three drafts.&lt;p&gt;What&amp;#x27;s even weirder is that Bard&amp;#x27;s first draft output ten lines of (wrong) Python code to calculate the result, even though my prompt mentioned nothing coding related. I wonder how non-technical users will react to this behavior. Another interesting thing is that the code follows Google&amp;#x27;s style guides.&lt;p&gt;[0]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=35893130&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=35893130&lt;/a&gt;</text></item></parent_chain><comment><author>bez_almighty</author><text>I couldn&amp;#x27;t replicate your results with that query on GPT-4.&lt;p&gt;Prompt: What’s heavier, a kilogram of steel or two kilograms of one kilogram feathers?&lt;p&gt;GPT-4: Two kilograms of one-kilogram feathers are heavier than a kilogram of steel. Despite the misconception caused by the popular question about what&amp;#x27;s heavier—a kilogram of steel or a kilogram of feathers (they are equal)—in this case, you are comparing two kilograms of feathers to one kilogram of steel. Hence, the feathers weigh more.</text></comment>