It has been shown that many important aspects of neural network learning take place within the very earliest iterations or epochs of training [1]}, [2]}. It is best to construct neural network training such that the network’s weight updates during the earliest epochs are relatively easy and of the highest quality for the task. This first part of a network’s training could use a curriculum learning approach. As the training proceeds, one increases the learning to span the full problem space and the hard work of learning to generalize is achieved during the middle epochs. The final epochs of the training should fine-tune the model on the desired data or tasks, because this is when the network learns the more complex patterns [3]} from the most relevant training samples.
