We distinguish our definition of the lottery ticket hypothesis from the weight rewinding technique [1]}. Lottery ticket hypothesis, on one hand, is a study of initialization state and network topology for a neural network, while weight rewinding, on the other hand, studies the trade-off between accuracy and subnetwork searching cost. Despite the difference, we can generalize the weight rewinding technique into the winning ticket identification principle, which is shown in Appendix . Detailed experimental evaluations of weight rewinding can also be found in Appendix .
