Recent PPNNs adopt a batch normalization layer [1]} after each activation layer to improve inference accuracy. Batch normalization first calculates the mean \(\mu \)  and the variance \(\sigma \)  of the input of each layer, and then normalizes the input as \(\overline{Y_{act}}\)  using \(\mu \) , \(\sigma \) , and learned parameters \(\gamma \)  and \(\beta \) .
\(\begin{split}Y_{batch} & = \gamma \overline{Y_{act}} +\beta = \gamma \frac{Y_{act}-\mu }{\sqrt{\sigma ^2+\epsilon }}+\beta \\Y_{batch} & = d Y_{act} +e\end{split}\) 
