We define general cyclical training as any collection of settings where the training starts and ends with “easy training” and the “hard training” happens during the middle epochs.
In other words, it can be considered as a combination of curriculum learning [1]} in the early epochs with fine-tuning toward the end of training, plus training over the full problem space for greater generalization happening during the middle epochs.
General cyclical training is analogous to curriculum learning in the sense that numerous specific techniques can embody its principles.
