Domain Temporal Alignment (Fig. REF , \(\blacksquare \)\(\square \) ).
We now consider enforcing the dynamic latent factors to be domain-invariant.
Similar to the image-based UDA, we propose to reduce the domain gaps between the cross-domain dynamic latent factors.
There are several ways to achieve this, and in this paper, we take advantage of the most popular adversarial-based idea [1]}.
Specifically, we build a domain classifier to discriminate whether the data is from the source domain or the target domain.
When back-propagating the gradients, a gradient reversal layer (GRL) is adopted to invert the gradients.
Like existing video-based UDA methods, we also conduct both frame-level and video-level alignments.
As TA\(^3\) N [2]} does, we exploit the temporal relation network (TRN) [3]} to discover the temporal relations among different frames, and then aggregate all the temporal relation features into the final video-level features.
This enables another level of alignment on the temporal relation features.
Thus, we have:
\( \mathcal {L}_{f} = \frac{1}{N} \sum _{i=1}^{N} \frac{1}{T} \sum _{t=1}^T CE \left[G_{f}(\mathbf {z}_{i\_t}^\mathcal {D}),d_i \right],\) 
\( \mathcal {L}_{r} = \frac{1}{N} \sum _{i=1}^{N} \frac{1}{T-1} \sum _{n=2}^{T} CE \left[ G_{r} \left( TrN_n (\mathbf {Z}_{i}^\mathcal {D}) \right) ,d_i \right],\) 
\( \mathcal {L}_{v} = \frac{1}{N} \sum _{i=1}^{N} CE \left[ G_{v} \left( \frac{1}{T-1} \sum _{n=2}^{T} TrN_n (\mathbf {Z}_{i}^\mathcal {D}) \right) ,d_i \right],\) 
