To address this problem, two methods, including temporal ensembling [1]} and mean teacher [2]}, were proposed to obtain a more stable target output \(f^{\prime }_{\theta }(x)\) . Specifically, temporal ensembling uses an exponentially accumulated average of outputs,
\(f_{\theta }(x)\) , to make the target output smooth and consistent. Inspired by this method, mean teacher instead uses a prediction function which is parametrized by an exponentially accumulated average of \(\theta \)  during the training.
Like the \(\pi \) -model, mean teacher adds a mean squared error loss \(d(f_{\theta }(x), f^{\prime }_{\theta }(x))\)  as a regularization term to the total loss function for training the network. It has been shown that mean teacher outperforms temporal ensembling in practice [2]}. Contrary to stochastic perturbation methods which rely on constructing \(f_{\theta }(x)\)  stochastically, VAT in the first step approximates a small perturbation \(r\)  to add it to \(x\)  which significantly changes the
prediction of the model \(f_{\theta }(x)\) . In the next step, a consistency regularization technique is applied to minimize \(d(f_{\theta }(x), f_{\theta }(x+r))\)  with respect to \(\theta \)  which is the parameters of the model.
