As legacy training and cycle training are both adopted in offline training phase, the input of template kernel in RPN is compatible with template feature extracted from patches or feature retrieved from historical search feature by CPT module.
For real-time tracking, our tracker can perform like SiamRPN [1]}, operating at a high speed about 80 FPS.
For robust tracking, cycle training enables the tracker to update with a memory queue.
Detailly, a memory queue of length \(N_L\)  stored with historical search features and region masks is maintained, including features of initial frame and \(N_L - 1\)  historical samples of the highest score. Once the memory queue is updated, the memory kernel will be updated. Besides, the hidden template \(H_t\)  is updated every \(N_s\)  frames with the highest score in this short interval.
For the trade-off between speed and accuracy, the classification map \(R_{cls}\)  from the legacy kernel and memory kernel is combined with \(R_{cls} = (1 - \lambda _m) R_{cls}^{L} + \lambda _m R_{cls}^{M}\) , \(\lambda _m\)  denotes the weight to balance the classification score, while the regression map is generated with the template kernel extracted from the initial template patch. Here, \(R_{cls}^{L}\)  denotes the response map in classification branch that applying the template kernel extracted from initial frame, while \(R_{cls}^{M}\)  is the classification response map when applying the template kernel extracted from online memory queue.
Specifically, We set \(N_L =6\)  and \(N_s=10\)  throughout all online updating experiments.
