<story><title>Self-supervised learning: The dark matter of intelligence</title><url>https://ai.facebook.com/blog/self-supervised-learning-the-dark-matter-of-intelligence/</url></story><parent_chain><item><author>montenegrohugo</author><text>One of the problems we have to be conscious about with self supervised learning is that the already existing problem of transparency and understandability gets worsened by orders of magnitude. If current billion parameter models are opaque, the result of SSL is, or will be, the equivalent of a black hole. How do you understand what the model has learned? Why has it learned that way? What things could have possible gone completely, horribly wrong in its quest to optimize its loss function?&lt;p&gt;This somewhat humorous but also worrying paper comes to mind:
&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;Lu56xVlZ40M&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;Lu56xVlZ40M&lt;/a&gt;&lt;p&gt;I think we really need to consider how we can mitigate the risk of raising completely ununderstandable yet scarily capable machines.</text></item></parent_chain><comment><author>iujjkfjdkkdkf</author><text>I dont agree with this, either we know what the pretext task was, or we fine tune with labels, but in either case, it&amp;#x27;s no different to analyze than supervised models.&lt;p&gt;I do agree that more parameters and a bigger training set makes the models more opaque (or at least more expensive to understand), but the self supervision is not the reason.</text></comment>
<story><title>Self-supervised learning: The dark matter of intelligence</title><url>https://ai.facebook.com/blog/self-supervised-learning-the-dark-matter-of-intelligence/</url></story><parent_chain><item><author>montenegrohugo</author><text>One of the problems we have to be conscious about with self supervised learning is that the already existing problem of transparency and understandability gets worsened by orders of magnitude. If current billion parameter models are opaque, the result of SSL is, or will be, the equivalent of a black hole. How do you understand what the model has learned? Why has it learned that way? What things could have possible gone completely, horribly wrong in its quest to optimize its loss function?&lt;p&gt;This somewhat humorous but also worrying paper comes to mind:
&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;Lu56xVlZ40M&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;Lu56xVlZ40M&lt;/a&gt;&lt;p&gt;I think we really need to consider how we can mitigate the risk of raising completely ununderstandable yet scarily capable machines.</text></item></parent_chain><comment><author>Forge36</author><text>I have this same problem with people. Just because we don&amp;#x27;t understand perfectly doesn&amp;#x27;t mean the output is not useful.</text></comment>