<story><title>OpenAI technical goals</title><url>https://openai.com/blog/openai-technical-goals/</url></story><parent_chain><item><author>astanway</author><text>Almost no one who actually works or has done serious research in ML is genuinely concerned about &amp;quot;malevolent AI&amp;quot;. We are so, so, so far away from anything remotely close to that. Please stop trying to gin up fear and listen to the experts, who uniformly agree that this is not something to be concerned about.</text></item><item><author>Houshalter</author><text>I&amp;#x27;m concerned that none of these goals involve AI safety. Instead all of their goals are nearly the exact opposite, of accelerating AI technology as much as possible.&lt;p&gt;Safety was one of the main goals they promoted when it was founded. 2 of the 4 authors listed have publicly spoken about their belief of AI as an existential risk.&lt;p&gt;I&amp;#x27;m not saying that a game playing AI is going to take over the world. But it does demonstrate the risk - we still have no idea how to control such an AI. We can train it to get high scores. But it won&amp;#x27;t want to do anything other than get high scores. And it will do whatever it takes to get the highest score possible, even if it means exploiting the game or hurting other players, or disobeying it&amp;#x27;s masters.&lt;p&gt;Now imagine they succeed in making smarter AIs. And their research spawns new research, which inspires new research, etc. Perhaps over several decades we could have AIs that are a lot more formidable than being able to play Pac Man. But we may still not have made any progress on the ability to control them.</text></item></parent_chain><comment><author>saltvedt</author><text>&amp;quot;AI: A Modern Approach (...) authors, Stuart Russell and Peter Norvig, devote significant space to AI dangers and Friendly AI in section 26.3, “The Ethics and Risks of Developing Artificial Intelligence.”&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;intelligence.org&amp;#x2F;2013&amp;#x2F;10&amp;#x2F;19&amp;#x2F;russell-and-norvig-on-friendly-ai&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;intelligence.org&amp;#x2F;2013&amp;#x2F;10&amp;#x2F;19&amp;#x2F;russell-and-norvig-on-fr...&lt;/a&gt;&lt;p&gt;In addition, there is the AI Open Letter, which is signed by many &amp;quot;who actually works or has done serious research in ML&amp;quot;, including Demis Hassabis and Yann LeCun. From the letter:&lt;p&gt;&amp;quot;We recommend expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial: our AI systems must do what we want them to do. &amp;quot;&lt;p&gt;&lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;futureoflife.org&amp;#x2F;ai-open-letter&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;futureoflife.org&amp;#x2F;ai-open-letter&amp;#x2F;&lt;/a&gt;&lt;p&gt;Are the experts concerned about a &amp;quot;Skynet scenario&amp;quot;? No. But there is certainly genuine concern from the experts.</text></comment>
<story><title>OpenAI technical goals</title><url>https://openai.com/blog/openai-technical-goals/</url></story><parent_chain><item><author>astanway</author><text>Almost no one who actually works or has done serious research in ML is genuinely concerned about &amp;quot;malevolent AI&amp;quot;. We are so, so, so far away from anything remotely close to that. Please stop trying to gin up fear and listen to the experts, who uniformly agree that this is not something to be concerned about.</text></item><item><author>Houshalter</author><text>I&amp;#x27;m concerned that none of these goals involve AI safety. Instead all of their goals are nearly the exact opposite, of accelerating AI technology as much as possible.&lt;p&gt;Safety was one of the main goals they promoted when it was founded. 2 of the 4 authors listed have publicly spoken about their belief of AI as an existential risk.&lt;p&gt;I&amp;#x27;m not saying that a game playing AI is going to take over the world. But it does demonstrate the risk - we still have no idea how to control such an AI. We can train it to get high scores. But it won&amp;#x27;t want to do anything other than get high scores. And it will do whatever it takes to get the highest score possible, even if it means exploiting the game or hurting other players, or disobeying it&amp;#x27;s masters.&lt;p&gt;Now imagine they succeed in making smarter AIs. And their research spawns new research, which inspires new research, etc. Perhaps over several decades we could have AIs that are a lot more formidable than being able to play Pac Man. But we may still not have made any progress on the ability to control them.</text></item></parent_chain><comment><author>Houshalter</author><text>The fact they aren&amp;#x27;t concerned is exactly the problem. They should be. And in fact many are. See the SSC link posted below (&lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;slatestarcodex.com&amp;#x2F;2015&amp;#x2F;05&amp;#x2F;22&amp;#x2F;ai-researchers-on-ai-risk&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;slatestarcodex.com&amp;#x2F;2015&amp;#x2F;05&amp;#x2F;22&amp;#x2F;ai-researchers-on-ai-ri...&lt;/a&gt;), or this survey (&lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;www.nickbostrom.com&amp;#x2F;papers&amp;#x2F;survey.pdf&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;www.nickbostrom.com&amp;#x2F;papers&amp;#x2F;survey.pdf&lt;/a&gt;). Certainly I am worried about it. There is far from &amp;quot;uniform agreement&amp;quot; that AI is safe. And appeals to authority would not comforting even if there was an authority to appeal to.</text></comment>