The weight rewinding technique [1]} finds that the accuracy of the subnetwork improves significantly when
using the weights at early stage of pretraining (i.e. \(\theta _{t}\)  where \(t<T\) ) as the initial point to train the subnetwork \(f(x; (\theta _{t}\odot m))\) . Since the weights have rewound to an early stage of pretraining, the training epochs for subnetworks will decrease by \(t\)  to ensure same training efforts (i.e. training subnetwork for \(T-t\)  epochs and arrives at \(f(x;(\theta _{t}\odot m)_{T-t})\) ). We define the following settings regarding the weight rewinding technique:
