HEX: Human-in-the-loop Explainability via Deep Reinforcement Learning
The use of machine learning (ML) models in decision-making contexts,
particularly those used in high-stakes decision-making, are fraught with issue
and peril since a person - not a machine - must ultimately be held accountable
for the consequences of the decisions made using such systems. Machine learning
explainability (MLX) promises to provide decision-makers with
prediction-specific rationale, assuring them that the model-elicited
predictions are made for the right reasons and are thus reliable. Few works
explicitly consider this key human-in-the-loop (HITL) component, however. In
this work we propose HEX, a human-in-the-loop deep reinforcement learning
approach to MLX. HEX incorporates 0-distrust projection to synthesize decider
specific explanation-providing policies from any arbitrary classification
model. HEX is also constructed to operate in limited or reduced training data
scenarios, such as those employing federated learning. Our formulation
explicitly considers the decision boundary of the ML model in question, rather
than the underlying training data, which is a shortcoming of many
model-agnostic MLX methods. Our proposed methods thus synthesize HITL MLX
policies that explicitly capture the decision boundary of the model in question
for use in limited data scenarios.