Manual differentiation [1]} is widely used in frameworks. It can greatly impact the efficiency of the element-wise operation. For example, Sigmoid is a classic activation function: \(S(x) = \frac{1}{1+e^{-x}}\)  and would produce lots of intermediate variables with the original format. Frameworks implement its enclosed symbolic format as a basic operator. Without loss of practicability, we define the symbolic operators implemented in frameworks as the primitive operation, including tanh, sigmoid and softplus.
