We introduce AI rationalization, an approach for generating explanations of autonomous system behavior as if a human had performed the behavior. We describe a rationalization technique that uses neural machine translation to translate internal state-action representations of an autonomous agent into natural language. We evaluate our technique in the Frogger game environment, training an autonomous game playing agent to rationalize its action choices using natural language. A natural language training corpus is collected from human players thinking out loud as they play the game. We motivate the use of rationalization as an approach to explanation generation and show the results of two experiments evaluating the effectiveness of rationalization. Results of these evaluations show that neural machine translation is able to accurately generate rationalizations that describe agent behavior, and that rationalizations are more satisfying to humans than other alternative methods of explanation.
Explainable AI has been explored in the context of ad-hoc techniques for transforming simulation logs to explanations @cite_4 , intelligent tutoring systems @cite_15 , transforming AI plans into natural language @cite_18 , and translating multiagent communication policies into natural language @cite_17 . Our work differs in that the generated rationalizations do not need to be truly representative of the algorithm's decision-making process. This is a novel way of applying explainable AI techniques to sequential decision-making in stochastic domains.
@cite_15 @cite_4 @cite_18 @cite_17
2962861173 2135583756 2906586541
We aim to produce predictive models that are not only accurate, but are also interpretable to human experts. Our models are decision lists, which consist of a series of if...then... statements (for example, if high blood pressure, then stroke) that discretize a high-dimensional, multivariate feature space into a series of simple, readily interpretable decision statements. We introduce a generative model called Bayesian Rule Lists that yields a posterior distribution over possible decision lists. It employs a novel prior structure to encourage sparsity. Our experiments show that Bayesian Rule Lists has predictive accuracy on par with the current top algorithms for prediction in machine learning. Our method is motivated by recent developments in personalized medicine, and can be used to produce highly accurate and interpretable medical scoring systems. We demonstrate this by producing an alternative to the CHADS2 score, actively used in clinical practice for estimating the risk of stroke in patients that have atrial fibrillation. Our model is as interpretable as CHADS2, but more accurate. Opponent behavior in today's computer games is often the result of a static set of Artificial Intelligence (AI) behaviors or a fixed AI script. While this ensures that the behavior is reasonably intelligent, it also results in very predictable behavior. This can have an impact on the replayability of entertainment-based games and the educational value of training-based games. This paper proposes a move away from static, scripted AI by using a combination of deliberative and reactive planning. The deliberative planning (or Strategic AI) system creates a novel strategy for the AI opponent before each gaming session. The reactive planning (or Tactical AI) system executes this strategy in real-time and adapts to the player and the environment. These two systems, in conjunction with a future automated director module, form the Adaptive Opponent Architecture. This paper describes the architecture and the details of the deliberative and reactive planning components. Several approaches have recently been proposed for learning decentralized deep multiagent policies that coordinate via a differentiable communication channel. While these policies are effective for many tasks, interpretation of their induced communication strategies has remained a challenge. Here we propose to interpret agents' messages by translating them. Unlike in typical machine translation problems, we have no parallel data to learn from. Instead we develop a translation model based on the insight that agent messages and natural language strings mean the same thing if they induce the same belief about the world in a listener. We present theoretical guarantees and empirical evidence that our approach preserves both the semantics and pragmatics of messages by ensuring that players communicating through a translation layer do not suffer a substantial loss in reward relative to players with a common language.