
Understanding user questions.
The framework is modular to enable different ways for communicating a question to the AI system. Depending on the user interface a question could be given in different forms, for example speech, visual gestures or text input. Different technologies can be used to translate the question into a formal question such as: speech recognition, Natural Language Processing methods or human body tracking. Context can play a crucial role in understanding the question that the user asks. [1]} showed an example of a question requiring two different explanations depending on the context in which it was asked. In each case, improperly interpreting the question lead to an unsatisfying explanation. One promising direction for addressing this challenge is the use of argumentation [2]}.

Formally categorising the set of questions that can be answered with contrastive explanations.
Although some philosophers, such as [3]}, noted that “why"-questions can be implicitly or explicitly understood as: “why is A better than some alternative?", there might be questions in the planning space for which contrastive explanations are not well-suited. For example, if the user is simply trying to understand the conditions or requirements for various actions in the plan, or the causal or temporal structure of the plan, a contrastive explanation may not be appropriate. Instead it might be more appropriate to highlight the causal structure in an abstracted version of the plan. An important issue for future work is the development of a formal taxonomy of the types of questions that should be addressed using contrastive explanations [4]}.

Expressing constraints on actions and plan structure.
A contrastive question requires creating a hypothetical planning model, which is often characterised by constraints on what actions are permissible in the plan and how they are arranged. For example, the question “Why did you use action A rather than action B for achieving P?” requires planning with the hypothetical model where B is required to be in the causal support for achieving P, but A is not in that causal support. This is substantially more difficult than just universally excluding A from the plan and forcing B into the plan because A or B might be required or prohibited elsewhere in the plan. Currently we do not have a good language for expressing these kinds of constraints. PDDL 3 allows the expression of simple constraints on the order in which goals are achieved, but does not have the ability to express constraints on action inclusion, exclusion, or ordering, and does not allow us to place more complex constraints on how something is achieved or on plan structure. We would like to be able to say something simple like “\(\mbox{Supports}(B, P) \wedge \lnot \mbox{Supports}(B, P)\) ”. LTL will likely play a key role in defining the semantics of any such language, but additional concepts concerning plan structure are needed, such as the ability to specify that an action is part of the causal support for a goal or subgoal.

Compiling constraints into the HModel. We showed examples of how a constraint derived from a user question could be compiled to form an HModel. However, providing compilations for more general constraints (like the one above) and ensuring their correctness is an important issue. Additionally, the compilation can lead to producing plans which might differ from the original plan in ways unrelated to the user question. We believe that the work on planning with preferences [5]} and state-trajectory constraints (Baier et al. 2009) is an important first step, but does not yet address the full range of constraints needed.

Forming and presenting contrastive explanations.
The form of contrastive explanation we provide, as discussed in Section 3.4 and shown in the GUI in Fig. REF , is a very simple one that presents the original plan and HPlan and highlights the action differences between them. Also, it is possible to obtain hierarchical contrastive explanations by asking consecutive questions. However, this does not show the causality of the plans, or the differences in their causal structure. Fig. REF  shows a possible composite causal representation for both the original plan and the HPlan, with the differences shown in different colors. This way of visualising the explanation can help to elucidate how the two plans achieve, or fail to achieve, the (sub)goals of the problem with respect to specific actions in the domain. However, for larger and more complex plans we expect that some form of abstraction will be necessary in order to effectively compare and contrast plans; the user might wants to see the important differences between two plans, not all the details. What counts as details remains an open research question, but is likely related to action costs and to the ease and importance of achieving various subgoals. [6]} have done some initial work in this area, and have considered milestones as important abstractions for purposes of explanation. The issues of what constitutes a good explanation, and how to visualize it or present it remain intertwined. Some synergy between researchers in planning, data visualization (e.g., [7]} or [8]}), and social sciences [9]} would be fruitful.
<FIGURE><FIGURE>
Providing explanations for complex questions. In the presented approach, a user is able to iteratively ask questions to refine the explanation. If the explanation does not satisfy the user, or the question they have is more complex, this approach can provide the user with a deeper understanding. However, this process could be automated by analysing a more complex question the user might have, and decomposing it into several formal questions. In this case new constraints can be added to the HModel automatically until the explanation addresses the intended question and potentially the context it was asked within.

Assessing the effectiveness of explanations.
We believe it is crucial to be able to acquire evidence of the effectiveness of an explanation. In particular, if engendering trust is the motivation for Explainable Planning and XAI in general, then we should look at the actual experience of the users and check whether they gain confidence in the planner or not.
For this, a vital step for planning researchers is to include user studies to assess the effectiveness of the explanations they are providing.

