Feature extraction [1]} is a set of techniques aiming to find inputs, known as features, that maximize the activation of one or more neurons. Usually, feature extraction is used as a method for model interpretability, where one seeks to understand a neural network by analyzing how much each neuron contributes for modelling the training data. The process of obtaining these features is, in a sense, an attempt towards reversing a neural network. Since a neural network is composed by functions which map inputs to outputs, a feature is the input we would have given a target activation for a group of posterior selected neurons.
