Laziness, Barren Plateau, and Noise in Machine Learning
We define \emph{laziness} to describe a large suppression of variational
parameter updates for neural networks, classical or quantum. In the quantum
case, the suppression is exponential in the number of qubits for randomized
variational quantum circuits. We discuss the difference between laziness and
\emph{barren plateau} in quantum machine learning created by quantum physicists
in \cite{mcclean2018barren} for the flatness of the loss function landscape
during gradient descent. We address a novel theoretical understanding of those
two phenomena in light of the theory of neural tangent kernels. For noiseless
quantum circuits, without the measurement noise, the loss function landscape is
complicated in the overparametrized regime with a large number of trainable
variational angles. Instead, around a random starting point in optimization,
there are large numbers of local minima that are good enough and could minimize
the mean square loss function, where we still have quantum laziness, but we do
not have barren plateaus. However, the complicated landscape is not visible
within a limited number of iterations, and low precision in quantum control and
quantum sensing. Moreover, we look at the effect of noises during optimization
by assuming intuitive noise models, and show that variational quantum
algorithms are noise-resilient in the overparametrization regime. Our work
precisely reformulates the quantum barren plateau statement towards a precision
statement and justifies the statement in certain noise models, injects new hope
toward near-term variational quantum algorithms, and provides theoretical
connections toward classical machine learning. Our paper provides conceptual
perspectives about quantum barren plateaus, together with discussions about the
gradient descent dynamics in \cite{together}.