Providing theoretical guarantees for neural networks is a known challenge. Pioneering work by [1]}, [2]}, [3]} analyzes the GE dynamics form the statistical mechanics perspective on teacher-student framework [4]} to understand the performance of neural networks on unseen test data. All our theoretical analyses throughout this work closely follow [5]}, [6]}, who analyzed results for the case where the student network has more number of hidden nodes than the teacher network.
