<story><title>We built a system like Apple’s to flag CSAM and concluded the tech was dangerous</title><url>https://www.washingtonpost.com/opinions/2021/08/19/apple-csam-abuse-encryption-security-privacy-dangerous/</url></story><parent_chain><item><author>knaik94</author><text>One additional issue that I haven&amp;#x27;t really seen discussed is how to handle a situation when a false accusation is made.&lt;p&gt;If a person knows the right people who work at these companies, things get sorted out, but I imagine sometimes a person is forced to just handle the consequences.&lt;p&gt;Stepping away from CSAM and going back to something like developer account and apps getting banned on platforms for violating vague &amp;quot;guidelines&amp;quot;. It&amp;#x27;s someone&amp;#x27;s livelihood that&amp;#x27;s sometimes destroyed. Demonetization, apps getting banned, payment processors freezing accounts are mostly black box events and most situations aren&amp;#x27;t even related to crimes dealing with CSAM.&lt;p&gt;If it was something the government made a mistake with, there&amp;#x27;s legal ways to fight for your rights. There&amp;#x27;s generally a level of transparency that is afforded to you.&lt;p&gt;It is concerning that people flagged for handling CSAM will not know if they have been manually reviewed. The need to keep the forwarding to authorities a secret is understandable, but a human review before forwarding is only necessary if you expect false positives to begin with. Keeping that flag secret seems like another black box you can&amp;#x27;t fight as a user.&lt;p&gt;I don&amp;#x27;t deny the value of catching these criminals, but it throws the idea of due process out the window when the only assurance so far has been &amp;quot;trust us to do the right thing&amp;quot;.&lt;p&gt;It&amp;#x27;s also weird how Apple has chosen to intentionally insert itself into the investigation pipeline rather than just let NCMEC handle it like all other cloud providers.&lt;p&gt;I am glad this hasn&amp;#x27;t flown under the radar just because it is Apple who is making these promises. I have heard non-tech people talk about this but there&amp;#x27;s a lot of misunderstanding.</text></item></parent_chain><comment><author>skinkestek</author><text>&amp;gt; It&amp;#x27;s also weird how Apple has chosen to intentionally insert itself into the investigation pipeline rather than just let NCMEC handle it like all other cloud providers.&lt;p&gt;Not so weird actually in my opinion:&lt;p&gt;Apple wants to make sure their customers doesn&amp;#x27;t get into trouble for no good reason, that would be bad for business.&lt;p&gt;Personally for me, a European living in Norway I do have great respect for the local police (mostly). Great people, mostly going out of their way to serve the public (again, mostly).&lt;p&gt;Norwegian child protection however is someone I&amp;#x27;d rather stay  clear of. They are well known to simultaneously ignore kids who need help and also harass innocent parents.&lt;p&gt;Again, not all of them are like this, many of them try to do good, but they seem to  a large extent to be working outside of control of the courts so if they mess up you have to go to a EU court to fix it. (Two or three cases just last 18 months from a small country like Norway.)&lt;p&gt;So something similar might also be at play, but I don&amp;#x27;t know what reputation NCMEC has, only that it is well known that a number of people have gotten serious trouble because of overeager employees at photo labs reporting innocent images.</text></comment>
<story><title>We built a system like Apple’s to flag CSAM and concluded the tech was dangerous</title><url>https://www.washingtonpost.com/opinions/2021/08/19/apple-csam-abuse-encryption-security-privacy-dangerous/</url></story><parent_chain><item><author>knaik94</author><text>One additional issue that I haven&amp;#x27;t really seen discussed is how to handle a situation when a false accusation is made.&lt;p&gt;If a person knows the right people who work at these companies, things get sorted out, but I imagine sometimes a person is forced to just handle the consequences.&lt;p&gt;Stepping away from CSAM and going back to something like developer account and apps getting banned on platforms for violating vague &amp;quot;guidelines&amp;quot;. It&amp;#x27;s someone&amp;#x27;s livelihood that&amp;#x27;s sometimes destroyed. Demonetization, apps getting banned, payment processors freezing accounts are mostly black box events and most situations aren&amp;#x27;t even related to crimes dealing with CSAM.&lt;p&gt;If it was something the government made a mistake with, there&amp;#x27;s legal ways to fight for your rights. There&amp;#x27;s generally a level of transparency that is afforded to you.&lt;p&gt;It is concerning that people flagged for handling CSAM will not know if they have been manually reviewed. The need to keep the forwarding to authorities a secret is understandable, but a human review before forwarding is only necessary if you expect false positives to begin with. Keeping that flag secret seems like another black box you can&amp;#x27;t fight as a user.&lt;p&gt;I don&amp;#x27;t deny the value of catching these criminals, but it throws the idea of due process out the window when the only assurance so far has been &amp;quot;trust us to do the right thing&amp;quot;.&lt;p&gt;It&amp;#x27;s also weird how Apple has chosen to intentionally insert itself into the investigation pipeline rather than just let NCMEC handle it like all other cloud providers.&lt;p&gt;I am glad this hasn&amp;#x27;t flown under the radar just because it is Apple who is making these promises. I have heard non-tech people talk about this but there&amp;#x27;s a lot of misunderstanding.</text></item></parent_chain><comment><author>hirvi74</author><text>&amp;gt; It&amp;#x27;s also weird how Apple has chosen to intentionally insert itself into the investigation pipeline rather than just let NCMEC handle it like all other cloud providers.&lt;p&gt;So, I am not much of conspiracy theorist, but I do like to sometimes fantasize about alternative realities in which they were true.&lt;p&gt;I am not saying the US government had any involvement in Apple&amp;#x27;s decision, but what if they did? I do agree with your point about how this topic more or less came out of Left-field. It&amp;#x27;s clear that Apple did not just recently acquire the technological ability to produce this feature in 2021. This feature could have been implemented years ago (like many other companies with a consumer-available cloud storage model already did to some degree). I am just curious if Apple did not really have a &amp;quot;choice&amp;quot; in this matter. Perhaps my monkey brain just want this to be the case.</text></comment>