Activation layers are assumed to be a feature detector stage of DNNs [1]}. FC and CONV layers provide linear representations of input data or it can be said that they work similarly to linear regressors and data transformed by these layers is considered to be at the feature extraction stage [2]}. Therefore, to learn non-linear features of data, an activation layer must be used after FC and CONV layers. In the activation layer, feature maps from previous layers go through an activation function to form an activation map. Some commonly used activation functions are sigmoid, hyperbolic tangent (tanh), rectified linear unit (ReLU), LiSHT [3]} and softmax. However, in HSI analysis, softmax and ReLU are widely employed activation functions [2]}. Figure REF  presents a graphical representation of a few commonly utilized activation functions.
<FIGURE>