GRETEL: A unified framework for Graph Counterfactual Explanation Evaluation
Machine Learning (ML) systems are a building part of the modern tools which
impact our daily life in several application domains. Due to their black-box
nature, those systems are hardly adopted in application domains (e.g. health,
finance) where understanding the decision process is of paramount importance.
Explanation methods were developed to explain how the ML model has taken a
specific decision for a given case/instance. Graph Counterfactual Explanations
(GCE) is one of the explanation techniques adopted in the Graph Learning
domain. The existing works of Graph Counterfactual Explanations diverge mostly
in the problem definition, application domain, test data, and evaluation
metrics, and most existing works do not compare exhaustively against other
counterfactual explanation techniques present in the literature. We present
GRETEL, a unified framework to develop and test GCE methods in several
settings. GRETEL is a highly extensible evaluation framework which promotes the
Open Science and the evaluations reproducibility by providing a set of
well-defined mechanisms to integrate and manage easily: both real and synthetic
datasets, ML models, state-of-the-art explanation techniques, and evaluation
measures. To present GRETEL, we show the experiments conducted to integrate and
test several synthetic and real datasets with several existing explanation
techniques and base ML models.