<story><title>Sora: Creating video from text</title><url>https://openai.com/sora</url></story><parent_chain><item><author>bamboozled</author><text>Wouldn&amp;#x27;t having a good understanding of physics mean you know that a women doesn&amp;#x27;t slide down the road when she walks? Wouldn&amp;#x27;t it know that a woolly mammoth doesn&amp;#x27;t emit profuse amounts steam when walking on frozen snow? Wouldn&amp;#x27;t the model know that legs are solid objects in which other object cannot pass through?&lt;p&gt;Maybe I&amp;#x27;m missing the big picture here, but the above and all the weird spatial errors, like miniaturization of people make me think you&amp;#x27;re wrong.&lt;p&gt;Clearly the model is an achievement and doing something interesting to produce these videos, and they are pretty cool, but understanding physics seems like quite a stretch?&lt;p&gt;I also don&amp;#x27;t really get the excitement about the girl on the train in Tokyo:&lt;p&gt;&lt;i&gt;In the Tokyo one, the model is smart enough to figure out that on a train, the reflection would be of a passenger, and the passenger has Asian traits since this is Tokyo&lt;/i&gt;&lt;p&gt;I don&amp;#x27;t know a lot about how this model works personally, but I&amp;#x27;m guessing in the training data the vast majority of people riding trains in Tokyo featured asian people in them, assuming this model works on statistics like all of the other models I&amp;#x27;ve seen recently from Open AI, then why is it interesting the girl in the reflection was Asian? Did you not expect that?</text></item><item><author>sebastiennight</author><text>I think the implications go much further than just the image&amp;#x2F;video considerations.&lt;p&gt;This model shows a very good (albeit not perfect) understanding of the physics of objects and relationships between them. The announcement mentions this several times.&lt;p&gt;The OpenAI blog post lists &amp;quot;Archeologists discover a generic plastic chair in the desert, excavating and dusting it with great care.&amp;quot; as one of the &amp;quot;failed&amp;quot; cases.
But this (and &amp;quot;Reflections in the window of a train traveling through the Tokyo suburbs.&amp;quot;) seem to me to be 2 of the most important examples.&lt;p&gt;- In the Tokyo one, the model is smart enough to figure out that on a train, the reflection would be of a passenger, and the passenger has Asian traits since this is Tokyo.
- In the chair one, OpenAI says the model failed to model the physics of the object (which hints that it did try to, which is not how the early diffusion models worked ; they just tried to generate &amp;quot;plausible&amp;quot; images). And we can see one of the archeologists basically chasing the chair down to grab it, which does correctly model the interaction with a floating object.&lt;p&gt;I think we can&amp;#x27;t underestimate how crucial that is to the building of a general model that has a strong model of the world. Not just a &amp;quot;theory of mind&amp;quot;, but a litteral understanding of &amp;quot;what will happen next&amp;quot;, independently of &amp;quot;what would a human say would happen next&amp;quot; (which is what the usual text-based models seem to do).&lt;p&gt;This is going to be much more important, IMO, than the video aspect.</text></item></parent_chain><comment><author>csomar</author><text>&amp;gt; Wouldn&amp;#x27;t having a good understanding of physics mean you know that a women doesn&amp;#x27;t slide down the road when she walks? Wouldn&amp;#x27;t it know that a woolly mammoth doesn&amp;#x27;t emit profuse amounts steam when walking on frozen snow? Wouldn&amp;#x27;t the model know that legs are solid objects in which other object cannot pass through?&lt;p&gt;This just hit me but humans do not have a good understanding of physics; or maybe most of humans have no understanding of physics. We just observe and recognize whether it&amp;#x27;s familiar or not.&lt;p&gt;AI will need to be, that being the case, way more powerful than a human mind. Maybe orders of magnitude more &amp;quot;neural networks&amp;quot; than a human brain has.</text></comment>
<story><title>Sora: Creating video from text</title><url>https://openai.com/sora</url></story><parent_chain><item><author>bamboozled</author><text>Wouldn&amp;#x27;t having a good understanding of physics mean you know that a women doesn&amp;#x27;t slide down the road when she walks? Wouldn&amp;#x27;t it know that a woolly mammoth doesn&amp;#x27;t emit profuse amounts steam when walking on frozen snow? Wouldn&amp;#x27;t the model know that legs are solid objects in which other object cannot pass through?&lt;p&gt;Maybe I&amp;#x27;m missing the big picture here, but the above and all the weird spatial errors, like miniaturization of people make me think you&amp;#x27;re wrong.&lt;p&gt;Clearly the model is an achievement and doing something interesting to produce these videos, and they are pretty cool, but understanding physics seems like quite a stretch?&lt;p&gt;I also don&amp;#x27;t really get the excitement about the girl on the train in Tokyo:&lt;p&gt;&lt;i&gt;In the Tokyo one, the model is smart enough to figure out that on a train, the reflection would be of a passenger, and the passenger has Asian traits since this is Tokyo&lt;/i&gt;&lt;p&gt;I don&amp;#x27;t know a lot about how this model works personally, but I&amp;#x27;m guessing in the training data the vast majority of people riding trains in Tokyo featured asian people in them, assuming this model works on statistics like all of the other models I&amp;#x27;ve seen recently from Open AI, then why is it interesting the girl in the reflection was Asian? Did you not expect that?</text></item><item><author>sebastiennight</author><text>I think the implications go much further than just the image&amp;#x2F;video considerations.&lt;p&gt;This model shows a very good (albeit not perfect) understanding of the physics of objects and relationships between them. The announcement mentions this several times.&lt;p&gt;The OpenAI blog post lists &amp;quot;Archeologists discover a generic plastic chair in the desert, excavating and dusting it with great care.&amp;quot; as one of the &amp;quot;failed&amp;quot; cases.
But this (and &amp;quot;Reflections in the window of a train traveling through the Tokyo suburbs.&amp;quot;) seem to me to be 2 of the most important examples.&lt;p&gt;- In the Tokyo one, the model is smart enough to figure out that on a train, the reflection would be of a passenger, and the passenger has Asian traits since this is Tokyo.
- In the chair one, OpenAI says the model failed to model the physics of the object (which hints that it did try to, which is not how the early diffusion models worked ; they just tried to generate &amp;quot;plausible&amp;quot; images). And we can see one of the archeologists basically chasing the chair down to grab it, which does correctly model the interaction with a floating object.&lt;p&gt;I think we can&amp;#x27;t underestimate how crucial that is to the building of a general model that has a strong model of the world. Not just a &amp;quot;theory of mind&amp;quot;, but a litteral understanding of &amp;quot;what will happen next&amp;quot;, independently of &amp;quot;what would a human say would happen next&amp;quot; (which is what the usual text-based models seem to do).&lt;p&gt;This is going to be much more important, IMO, than the video aspect.</text></item></parent_chain><comment><author>pests</author><text>They could test this by trying to generate the same image but set in New York, etc. I bet it would still be asain.</text></comment>