Existing learning rate schedules that do not require specification of the optimization stopping stepare greatly out-performed by learning rate schedules that depend on. We propose an approach that avoids the need for this stopping time by eschewing the use of schedules entirely, while exhibiting state-of-the-art performance compared to schedules across a wide family of problems ranging from convex problems to large-scale deep learning problems. OurSchedule-Freeapproach introduces no additional hyper-parameters over standard optimizers with momentum. Our method is a direct consequence of a new theory we develop that unifies scheduling and iterate averaging. An open source implementation of our method is available111https://github.com/facebookresearch/schedule_free.