As shown in Fig. REF , a computational neuron consists of
two stages in cascade: 1) an affine transformation that maps an
\(n\) -dimensional input vector to a scalar, and 2) a nonlinear activation
operator. The affine transformation is relatively easy to understand.
Yet, nonlinear activation obscures the function of a computational
neuron. If the function of nonlinear activation is well understood, one
may remove it and compensate its function with another mechanism. As
explained in Sec. REF , one neural network achieves two
objectives simultaneously [1]}: 1) finding an
expressive embedding of the input data, and 2) making decision (i.e.,
classification or regression) based on data embedding.
