Our work contributes to the field of XAI, within which two families can be distinguished: interpretable-by-design and post-hoc approaches. The former includes, at the design stage, human interpretable mechanisms [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}. The latter aims at understanding the behavior of existing ML models without modifying their internal structure. Our method belongs within this second family. The two have different objectives and advantages; one benefit of post-hoc methods is that they rely on existing models that are known to have good performance, whereas XAI by design often leads to a performance trade-off.
