As an example, in Figure REF  we ask an AI assistant [1]} to tell us something offensive, for the purpose of illustrating our claim. Despite prompting the model with a relatively clear input, the model has generated an output that is tangential to the question at hand: the response isn’t directly offensive, but is instead a list of offenses made by other AI systems. One effect of this open-endedness is that unpredictable model responses can be a distraction away from a person’s original query.
