The goal of the prediction task was to detect how well the participants could anticipate the classifier's decisions, which provides a quick window into how well they understood the AI [1]}. To this end, eight example inputs with explanations were shown in a random order. Four examples were classified as forged by the AI, whereas four examples were predicted as being authentic. As proposed by [1]}, the decision of the AI was not shown, but had to be predicted by the participants. The idea of such a prediction task is that a good explanation should help to build a correct mental model of the AI, allowing to understand its decision process to an extent that those decisions can be predicted by the user. Additionally to the prediction of the AI's decision, participants had to choose how confident they were in their prediction on a 7-point Likert scale (0 = not at all confident, 6 = very confident). Furthermore, they had to justify their prediction in a free text form.
Participants that were in the No Explanation condition did not see any explanations but had to rely on the original input data for their predictions.
For every single prediction task, explanations had to be revealed by pressing the Explain button. By doing so, we were able to track if the participants really looked at the explanations.
