<story><title>It looks like you’re trying to take over the world</title><url>https://www.gwern.net/Clippy</url></story><parent_chain><item><author>est31</author><text>These stories about AIs taking over the world in order to maximize a score often contain some computer that escapes the lab or such, in order to maximize some score. And indeed, it is true that taking over the world helps you to maximize whatever score you want to maximize.&lt;p&gt;But mankind, as a whole, is a clippy optimizer already, albeit a manual one. Right now it is &lt;i&gt;us&lt;/i&gt; that is destroying life around us, replacing rainforest with cattle farms and suburban developments. We are already working on farming robots to serve our goals. Does the lizard care if it is crushed by a farming robot&amp;#x27;s wheel or an AI tank&amp;#x27;s chain?&lt;p&gt;We are not the strongest, nor the biggest, nor the fastest, nor the best hearing, or the best seeing species. Still, we are the apex predator of this planet, thanks to our sole distinguishing feature, which is our intelligence and ability to cooperate. And we don&amp;#x27;t &lt;i&gt;realize&lt;/i&gt; it because economic growth seems &lt;i&gt;normal&lt;/i&gt;.&lt;p&gt;These clippy stories speak of a not unfounded fear that development might not stop with us, that one day some life form spawns that is even smarter than us, and turns us, the predators, into irrelevant side pieces that need to get out of the way for fulfilling the objective of the more powerful life form, even if it was us who brought that life form into existence or gave it that objective.&lt;p&gt;The forests of today are not inhabited by the life form that first discovered photosynthesis, but instead by highly specialized structures that grow higher so that they can put their competition into the shade. Similarly, just because we are the first highly intelligent + cooperative life doesn&amp;#x27;t mean we will be the top of the world forever.&lt;p&gt;I&amp;#x27;m not saying this with a light heart, I do feel sorry for the future humans, some of whom might even be alive today. But &amp;quot;how to contain reward optimizers&amp;quot; is a tough question, especially due to us running a giant project of reward optimization ourselves.</text></item></parent_chain><comment><author>ansible</author><text>&amp;gt; &lt;i&gt;We are not the strongest, nor the biggest, nor the fastest, nor the best hearing, or the best seeing species.&lt;/i&gt;&lt;p&gt;That is not strictly true.  Even leaving aside the tools argument (a human on a bicycle is vastly more energy efficient over long distances than a wildebeest, for example), our sight and hearing are actually pretty good compared to the rest of the animal kingdom.&lt;p&gt;For a certain range of tasks, we have the best visual perception of any species. Our eyes themselves are pretty damn good for close-in and medium range work. The pattern recognition processing that input is very, very good due to our big brains.  Tasks like being able to detect a tiger moving through the underbrush. Color vision is also very good (at least in daylight) to better pick out ripe fruit and such.&lt;p&gt;Our directional hearing is better than owls who hunt at night for a living. We can better measure the angle at which a sound source is coming from with around twice the accuracy of an owl. Again due to big brains and very complicated audio processing. The folds in your ears are there for a reason. We aren&amp;#x27;t as sensitive as other creatures like deer, who can steer their big ears (basically parabolic horns) in a particular direction.&lt;p&gt;And hand-eye coordination is far above any other species, even other primates.&lt;p&gt;Don&amp;#x27;t sell yourself short!</text></comment>
<story><title>It looks like you’re trying to take over the world</title><url>https://www.gwern.net/Clippy</url></story><parent_chain><item><author>est31</author><text>These stories about AIs taking over the world in order to maximize a score often contain some computer that escapes the lab or such, in order to maximize some score. And indeed, it is true that taking over the world helps you to maximize whatever score you want to maximize.&lt;p&gt;But mankind, as a whole, is a clippy optimizer already, albeit a manual one. Right now it is &lt;i&gt;us&lt;/i&gt; that is destroying life around us, replacing rainforest with cattle farms and suburban developments. We are already working on farming robots to serve our goals. Does the lizard care if it is crushed by a farming robot&amp;#x27;s wheel or an AI tank&amp;#x27;s chain?&lt;p&gt;We are not the strongest, nor the biggest, nor the fastest, nor the best hearing, or the best seeing species. Still, we are the apex predator of this planet, thanks to our sole distinguishing feature, which is our intelligence and ability to cooperate. And we don&amp;#x27;t &lt;i&gt;realize&lt;/i&gt; it because economic growth seems &lt;i&gt;normal&lt;/i&gt;.&lt;p&gt;These clippy stories speak of a not unfounded fear that development might not stop with us, that one day some life form spawns that is even smarter than us, and turns us, the predators, into irrelevant side pieces that need to get out of the way for fulfilling the objective of the more powerful life form, even if it was us who brought that life form into existence or gave it that objective.&lt;p&gt;The forests of today are not inhabited by the life form that first discovered photosynthesis, but instead by highly specialized structures that grow higher so that they can put their competition into the shade. Similarly, just because we are the first highly intelligent + cooperative life doesn&amp;#x27;t mean we will be the top of the world forever.&lt;p&gt;I&amp;#x27;m not saying this with a light heart, I do feel sorry for the future humans, some of whom might even be alive today. But &amp;quot;how to contain reward optimizers&amp;quot; is a tough question, especially due to us running a giant project of reward optimization ourselves.</text></item></parent_chain><comment><author>slibhb</author><text>&amp;gt; But mankind, as a whole, is a clippy optimizer already, albeit a manual one. Right now it is us that is destroying life around us, replacing rainforest with cattle farms and suburban developments. We are already working on farming robots to serve our goals. Does the lizard care if it is crushed by a farming robot&amp;#x27;s wheel or an AI tank&amp;#x27;s chain?&lt;p&gt;We already have a built-in solution for this problem: biophilia. People like nature. Once we stop being poor, preserving nature, living in natural environments with trees and animals becomes a high priority. The people &amp;quot;cutting down the rain forest&amp;quot; will stop once they&amp;#x27;re sufficiently comfortable to prioritize nature over avoiding privation just like the Europeans stopped.&lt;p&gt;We used to built boats out of wood, now we build them out of metal. There&amp;#x27;s no reason we can&amp;#x27;t replace technologies that conflict with our biophilia with those that don&amp;#x27;t.</text></comment>