Progress with Stochastic Gradient Descent part4(Machine Learning)
Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent(arXiv) Author : : Avrajit Ghosh, He Lyu, Xitong Zhang, Rongrong Wang Abstract : It is well known that the finite step-size (h) in Gradient Descent (GD) implicitly regularizes solutions to flatter minima. A natural question to ask is “Does the momentum…
Progress with Stochastic Gradient Descent part4(Machine Learning) Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent(arXiv) Author : : Avrajit Ghosh, He Lyu, Xitong Zhang, Rongrong Wang Abstract : It is well known that the finite step-size (h) in Gradient Descent (GD) implicitly regularizes solutions to flatter minima. A natural question to ask is “Does the momentum… deep-learning
Progress Stochastic Gradient Descent part4Machine Learning Implicit regularization Heavyball momentum accelerated stochastic gradient descentarXiv Author Avrajit Ghosh Lyu Xitong Zhang Rongrong Wang Abstract well known finite stepsize h Gradient Descent GD implicitly regularizes solution flatter minimum natural question ask “Does momentum… deeplearning