<story><title>Employers’ Use of AI Tools Can Violate the Americans with Disabilities Act</title><url>https://www.justice.gov/opa/pr/justice-department-and-eeoc-warn-against-disability-discrimination</url></story><parent_chain></parent_chain><comment><author>theptip</author><text>One point that I think is under-discussed in the AI bias area:&lt;p&gt;While it is true that using an algorithmic process to select candidates may introduce discrimination against protected groups, it seems to me that it should be much easier to detect and prove than with previous processes with human judgement in the loop.&lt;p&gt;You can just subpoena the algorithm and then feed test data to it, and make observations. Even feed synthetic data like swapping in “stereotypically black” names for real resumes of other races, or in this case adding “uses a wheelchair” to a resume. (Of course in practice it’s more complex but hopefully this makes the point.)&lt;p&gt;With a human, you can’t really do an A&amp;#x2F;B test to determine if they would have prioritized a candidate if they hadn’t included some signal; it’s really easy to rationalize away discrimination at the margins.&lt;p&gt;So while most AI&amp;#x2F;ML developers are not currently strapping their models to a discrimination-tester, I think the end-state could be much better when they do.&lt;p&gt;(I think a concrete solution would be to regulate these models to require a certification with some standardized test framework to show that developers have actually attempted to control these potential sources of bias. Google has done some good work in this area: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;ai.google&amp;#x2F;responsibilities&amp;#x2F;responsible-ai-practices&amp;#x2F;?category=fairness&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;ai.google&amp;#x2F;responsibilities&amp;#x2F;responsible-ai-practices&amp;#x2F;...&lt;/a&gt; - though there is nothing stopping model-sellers from self-regulating and publishing this testing first, to try to get ahead of formal regulation.)</text></comment>
<story><title>Employers’ Use of AI Tools Can Violate the Americans with Disabilities Act</title><url>https://www.justice.gov/opa/pr/justice-department-and-eeoc-warn-against-disability-discrimination</url></story><parent_chain></parent_chain><comment><author>theptip</author><text>The linked article gives some examples that I think are very useful clarifications:&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.eeoc.gov&amp;#x2F;tips-workers-americans-disabilities-act-and-use-software-algorithms-and-artificial-intelligence&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.eeoc.gov&amp;#x2F;tips-workers-americans-disabilities-act...&lt;/a&gt;&lt;p&gt;&amp;gt; The format of the employment test can screen out people with disabilities [for example:] A job application requires a timed math test using a keyboard. Angela has severe arthritis and cannot type quickly.&lt;p&gt;&amp;gt; The scoring of the test can screen out people with disabilities [for example:] An employer uses a computer program to test “problem-solving ability” based on speech patterns for a promotion. Sasha meets the requirements for the promotion. Sasha stutters so their speech patterns do not match what the computer program expects.&lt;p&gt;Interestingly, I think the second one is problematic for common software interview practices. If your candidate asked for an accommodation (say, no live rapid-fire coding) due to a recognized medical condition, you would be legally required to provide it.&lt;p&gt;This request hasn’t come up for me in all the (startup) hiring I’ve done, but it could be tough to honor this request fairly on short notice, so worth thinking about in advance.</text></comment>