TITLE: Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms 

ABSTRACT: An increasing number of reports raise concerns about the risk that machine
learning algorithms could amplify health disparities due to biases embedded in
the training data. Seyyed-Kalantari et al. find that models trained on three
chest X-ray datasets yield disparities in false-positive rates (FPR) across
subgroups on the 'no-finding' label (indicating the absence of disease). The
models consistently yield higher FPR on subgroups known to be historically
underserved, and the study concludes that the models exhibit and potentially
even amplify systematic underdiagnosis. We argue that the experimental setup in
the study is insufficient to study algorithmic underdiagnosis. In the absence
of specific knowledge (or assumptions) about the extent and nature of the
dataset bias, it is difficult to investigate model bias. Importantly, their use
of test data exhibiting the same bias as the training data (due to random
splitting) severely complicates the interpretation of the reported disparities.
An increasing number of reports raise concerns about the risk that machine
learning algorithms could amplify health disparities due to biases embedded in
the training data. Seyyed-Kalantari et al. find that models trained on three
chest X-ray datasets yield disparities in false-positive rates (FPR) across
subgroups on the 'no-finding' label (indicating the absence of disease). The
models consistently yield higher FPR on subgroups known to be historically
underserved, and the study concludes that the models exhibit and potentially
even amplify systematic underdiagnosis. We argue that the experimental setup in
the study is insufficient to study algorithmic underdiagnosis. In the absence
of specific knowledge (or assumptions) about the extent and nature of the
dataset bias, it is difficult to investigate model bias. Importantly, their use
of test data exhibiting the same bias as the training data (due to random
splitting) severely complicates the interpretation of the reported disparities.
Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms
http://arxiv.org/abs/2201.07856v2