The intuition for cyclical softmax temperature is that in the first epochs, more confident predictions will help start moving the network's weights in the right direction. However, in the middle and later epochs, softer probabilities are more appropriate for the remaining training samples. In addition, softer probabilities reduce the confidence miscalibration of the predictions [1]} so that the network's training is based on losses that better match reality.
