Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
Recent years have seen the development of many open-source ML fairness
toolkits aimed at helping ML practitioners assess and address unfairness in
their systems. However, there has been little research investigating how ML
practitioners actually use these toolkits in practice. In this paper, we
conducted the first in-depth empirical exploration of how industry
practitioners (try to) work with existing fairness toolkits. In particular, we
conducted think-aloud interviews to understand how participants learn about and
use fairness toolkits, and explored the generality of our findings through an
anonymous online survey. We identified several opportunities for fairness
toolkits to better address practitioner needs and scaffold them in using
toolkits effectively and responsibly. Based on these findings, we highlight
implications for the design of future open-source fairness toolkits that can
support practitioners in better contextualizing, communicating, and
collaborating around ML fairness efforts.