<story><title>Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots</title><url>https://www.aclu.org/blog/privacy-technology/surveillance-technologies/amazons-face-recognition-falsely-matched-28</url></story><parent_chain><item><author>dfundako</author><text>I think a fun thought exercise is finding the fine line between tech and guns&amp;#x2F;alcohol&amp;#x2F;cars. You cannot sue a gun&amp;#x2F;car&amp;#x2F;alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can be used improperly and have disastrous results, like a car or gun), could Amazon or a tech administering it be liable for someone falsely imprisoned?</text></item><item><author>alphabettsy</author><text>Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those are real considerations for any program that has the potential to help ruin someone’s life.&lt;p&gt;It also possibly highlights the fact that algorithms are not immune to bias when they are designed by humans. Obviously I don’t think this bias is intentional, but so much of it isn’t and happens anyway.</text></item></parent_chain><comment><author>guitarbill</author><text>It&amp;#x27;s still ethically questionable. In fact I&amp;#x27;m struggling to come up with a better example than facial recognition tech (except other mass surveillance). Maybe cutting corners while developing driverless cars that results in the death of a pedestrian.&lt;p&gt;Almost every engineering discipline has a code of ethics [0][1][2][3]. It&amp;#x27;s time software &amp;quot;engineering&amp;quot; grew up and did the same.&lt;p&gt;I rarely see ethics mentioned on HN, and granted, people&amp;#x27;s view differ. But it&amp;#x27;s weird we&amp;#x27;re not having that conversation at all.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.raeng.org.uk&amp;#x2F;policy&amp;#x2F;engineering-ethics&amp;#x2F;ethics&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.raeng.org.uk&amp;#x2F;policy&amp;#x2F;engineering-ethics&amp;#x2F;ethics&lt;/a&gt;&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.ieee.org&amp;#x2F;about&amp;#x2F;corporate&amp;#x2F;governance&amp;#x2F;p7-8.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.ieee.org&amp;#x2F;about&amp;#x2F;corporate&amp;#x2F;governance&amp;#x2F;p7-8.html&lt;/a&gt;&lt;p&gt;[2] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.nspe.org&amp;#x2F;resources&amp;#x2F;ethics&amp;#x2F;code-ethics&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.nspe.org&amp;#x2F;resources&amp;#x2F;ethics&amp;#x2F;code-ethics&lt;/a&gt;&lt;p&gt;[3] &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;www.asce.org&amp;#x2F;code-of-ethics&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;www.asce.org&amp;#x2F;code-of-ethics&amp;#x2F;&lt;/a&gt;</text></comment>
<story><title>Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots</title><url>https://www.aclu.org/blog/privacy-technology/surveillance-technologies/amazons-face-recognition-falsely-matched-28</url></story><parent_chain><item><author>dfundako</author><text>I think a fun thought exercise is finding the fine line between tech and guns&amp;#x2F;alcohol&amp;#x2F;cars. You cannot sue a gun&amp;#x2F;car&amp;#x2F;alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can be used improperly and have disastrous results, like a car or gun), could Amazon or a tech administering it be liable for someone falsely imprisoned?</text></item><item><author>alphabettsy</author><text>Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those are real considerations for any program that has the potential to help ruin someone’s life.&lt;p&gt;It also possibly highlights the fact that algorithms are not immune to bias when they are designed by humans. Obviously I don’t think this bias is intentional, but so much of it isn’t and happens anyway.</text></item></parent_chain><comment><author>vorpalhex</author><text>&amp;gt; could Amazon or a tech administering it be liable for someone falsely imprisoned?&lt;p&gt;I would wager &amp;quot;absolutely not&amp;quot; for the exact same issue as firearms or cars. The person who mis-applied the technology or a middle man vendor though? Unless they fall under qualified immunity, there&amp;#x27;s your fall guy.</text></comment>