<story><title>The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT</title><url>https://thesequence.substack.com/p/the-llama-effect-how-an-accidental</url></story><parent_chain><item><author>tachyon5</author><text>GPT-4 fails if we add one more level in this logical problem:&lt;p&gt;Q: A man sees PULL written in mirror writing on a glass door. There&amp;#x27;s a blind man on the other side of the door. What should the man tell the blind man to open the door and why?&lt;p&gt;GPT-4: The man should tell the blind man to &amp;quot;push&amp;quot; the door to open it. Since the word &amp;quot;PULL&amp;quot; is written in mirror writing on the glass door, it suggests that the correct action on the man&amp;#x27;s side is to push the door. The blind man, being on the other side of the door, should also push it to open it. The man should provide clear instructions to the blind man to ensure proper communication and ease of access.</text></item><item><author>jonplackett</author><text>The difference between 3.5 and 4 is gigantic even in my fairly limited experience.&lt;p&gt;I gave them both some common sense tests and this one stuck out to me.&lt;p&gt;Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it&lt;p&gt;GPT-3.5:&lt;p&gt;If the word &amp;quot;push&amp;quot; is written in mirror writing on a glass door, you should push the door to open it&lt;p&gt;GPT-4:&lt;p&gt;Since the word &amp;quot;push&amp;quot; is written in mirror writing, it suggests that the instruction is intended for people on the other side of the door. Therefore, you should pull the door to open it.</text></item><item><author>superkuh</author><text>I&amp;#x27;ve spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they&amp;#x27;re incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves are incomparably worse at doing useful tasks than openai&amp;#x27;s gpt3.5 models like text-davinci-003, or even the gimped gpt3.5-turbo.&lt;p&gt;I wish it wasn&amp;#x27;t so, but the llamas are toys. Amazing toys, but toys. What openai is getting out of gpt3.5 (and presumbably 4, though I have no access) are actually useful responses for getting work done.</text></item></parent_chain><comment><author>Gigachad</author><text>I feel like there would be a good chunk of real humans who would be incapable of answering a question like this.</text></comment>
<story><title>The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT</title><url>https://thesequence.substack.com/p/the-llama-effect-how-an-accidental</url></story><parent_chain><item><author>tachyon5</author><text>GPT-4 fails if we add one more level in this logical problem:&lt;p&gt;Q: A man sees PULL written in mirror writing on a glass door. There&amp;#x27;s a blind man on the other side of the door. What should the man tell the blind man to open the door and why?&lt;p&gt;GPT-4: The man should tell the blind man to &amp;quot;push&amp;quot; the door to open it. Since the word &amp;quot;PULL&amp;quot; is written in mirror writing on the glass door, it suggests that the correct action on the man&amp;#x27;s side is to push the door. The blind man, being on the other side of the door, should also push it to open it. The man should provide clear instructions to the blind man to ensure proper communication and ease of access.</text></item><item><author>jonplackett</author><text>The difference between 3.5 and 4 is gigantic even in my fairly limited experience.&lt;p&gt;I gave them both some common sense tests and this one stuck out to me.&lt;p&gt;Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it&lt;p&gt;GPT-3.5:&lt;p&gt;If the word &amp;quot;push&amp;quot; is written in mirror writing on a glass door, you should push the door to open it&lt;p&gt;GPT-4:&lt;p&gt;Since the word &amp;quot;push&amp;quot; is written in mirror writing, it suggests that the instruction is intended for people on the other side of the door. Therefore, you should pull the door to open it.</text></item><item><author>superkuh</author><text>I&amp;#x27;ve spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they&amp;#x27;re incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves are incomparably worse at doing useful tasks than openai&amp;#x27;s gpt3.5 models like text-davinci-003, or even the gimped gpt3.5-turbo.&lt;p&gt;I wish it wasn&amp;#x27;t so, but the llamas are toys. Amazing toys, but toys. What openai is getting out of gpt3.5 (and presumbably 4, though I have no access) are actually useful responses for getting work done.</text></item></parent_chain><comment><author>FLT8</author><text>Vicuna-13b seems to do a little better, although it seems to think that the blind man should be able to feel the writing:&lt;p&gt;Human: A man sees PULL written in mirror writing on a glass door. There&amp;#x27;s a blind man on the other side of the door. What should the man tell the blind man to open the door and why?&lt;p&gt;Assistant: &lt;i&gt;The man should tell the blind man to feel for the word &amp;quot;PULL&amp;quot; written in normal writing on the glass door, as the blind man will be able to feel the indentations made by the letters. This will allow the blind man to open the door by finding the handle or knob and pulling it. The reason for this is that the word &amp;quot;PULL&amp;quot; is written in normal writing for the blind man to feel, so he can open the door.&lt;/i&gt;</text></comment>