We also conduct experiment on 2D feature encoder the RGB branch of TSN [1]} but fail. Similar result is also reported in [2]}. Since the RGB branch of TSN operates only on a single frame, it fails in catching the motion to represent temporal information. Instead, we deploy two popular 3D spatiotemporal feature encoders, C3D and I3D, whose results well-verified the capacity of MIST.
