Several works have been done on the area of feature extraction, specially applied for model interpretability and explainability [1]}, [2]}, [3]}. These techniques, used for extracting features, usually consist in activation maximization [4]}, where a group of neurons, which can involve from a single neuron up to an entire layer (or channel for convolutions), is selected to extract the feature by maximising its activation.
