An explanation can be viewed from a constraint satisfaction problem perspective; can a model satisfy all constraints, and if not, why not? Satisfiability-based formal methods, i.e. SAT, SMT, MILP, provide these types of proofs for abstract mathematical representation of trained models. The explanations are instances of constraint satisfaction problems. Formal methods are on increasing interest in providing explanations for model behaviors [1]}. Formal methods have also been used to assess the quality of explainable AI methods  [2]}.
