In this work, we employ Temporal Cycle Consistency (TCC) [1]} loss to learn temporal alignment. TCC optimizes for alignment by learning an embedding space that maximizes one-to-one nearest neighbour mappings between sequences. This is achieved through a loss that maximizes for cycle-consistent nearest neighbours given a pair of video sequences. In our case, the cycle consistency is applied on the graph abstraction instead of image features as done in the aforementioned video alignment methods. Specifically, given \(D^{\prime }\) , we sample a pair of bounding box sequences \(V^{\prime }_i = \lbrace I^{\prime i}_1, \dots , I^{\prime i}_{m_i} \rbrace \)  and \(V^{\prime }_j = \lbrace I^{\prime }{j}_1, \dots , I^{\prime j}_{m_j} \rbrace \)  and extract embeddings by applying the spatial interaction encoder defined in Equation REF . Thus, we obtain the encoded features \(S_i = \lbrace \psi ( I^{\prime i}_1), \dots , \psi (I^{\prime i}_{m_i})\rbrace \)  and \(S_j = \lbrace \psi ( I^{\prime j}_1), \dots , \psi (I^{\prime j}_{m_j})\rbrace \) . For the \(n\) th element in \(S_i\) , we first compute its nearest neighbour, \(\upsilon ^n_{ij}\) , in \(S_j\)  and then compute the probability that it cycles-back to the \(k\) th frame in \(S_i\)  as:
\(\beta ^k_{ijn} = \frac{e^{-||\upsilon ^n _{ij} - S_i^k ||^2}}{{\sum _k^{m_j} e^ {-|| \upsilon ^n_{ij} - S^k_i || ^2}}}\,,\upsilon ^n_{ij} = \sum _k^{m_j} \alpha _k s^k_j\,,\alpha _k = \frac{e^{-|| S^n_i - S^k_j ||^2}}{{\sum _k^{m_j} e^ {-|| S^n_i - S^k_j || ^2}}}\,.\) 
