188To Trust or to Think: Cognitive Forcing Functions Can
Reduce Overreliance on AI in AI-assisted Decision-making
ZANA BUÇINCA, 
To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
People supported by AI-powered decision support tools frequently overrely on
the AI: they accept an AI's suggestion even when that suggestion is wrong.
Adding explanations to the AI decisions does not appear to reduce the
overreliance and some studies suggest that it might even increase it. Informed
by the dual-process theory of cognition, we posit that people rarely engage
analytically with each individual AI recommendation and explanation, and
instead develop general heuristics about whether and when to follow the AI
suggestions. Building on prior research on medical decision-making, we designed
three cognitive forcing interventions to compel people to engage more
thoughtfully with the AI-generated explanations. We conducted an experiment
(N=199), in which we compared our three cognitive forcing designs to two simple
explainable AI approaches and to a no-AI baseline. The results demonstrate that
cognitive forcing significantly reduced overreliance compared to the simple
explainable AI approaches. However, there was a trade-off: people assigned the
least favorable subjective ratings to the designs that reduced the overreliance
the most. To audit our work for intervention-generated inequalities, we
investigated whether our interventions benefited equally people with different
levels of Need for Cognition (i.e., motivation to engage in effortful mental
activities). Our results show that, on average, cognitive forcing interventions
benefited participants higher in Need for Cognition more. Our research suggests
that human cognitive motivation moderates the effectiveness of explainable AI
solutions.
Zana Buçinca Maja Barbara Malaya Krzysztof Z. Gajos