Improving Model Understanding and Trust with Counterfactual Explanations of Model Confidence
In this paper, we show that counterfactual explanations of confidence scores
help users better understand and better trust an AI model's prediction in
human-subject studies. Showing confidence scores in human-agent interaction
systems can help build trust between humans and AI systems. However, most
existing research only used the confidence score as a form of communication,
and we still lack ways to explain why the algorithm is confident. This paper
also presents two methods for understanding model confidence using
counterfactual explanation: (1) based on counterfactual examples; and (2) based
on visualisation of the counterfactual space.