In light of the above motivations, the main objective of this paper is twofold. First, we aim to introduce and investigate the concept of flow factorization, which separates a raw optical flow map into an ego flow map and an object flow map. An ego flow map is a direct result of the motion of the viewpoint, and thus provides clues about the translation and rotation of the viewpoint. On the other hand, an object flow map is caused by the actual movements of corresponding objects between two different image frames, and thus can serve as an indicator of the speed and direction of moving objects in a scene. Since these two types of factorized flow maps bear different physical meanings, directly leveraging raw flow maps (i.e., the flow maps directly reflecting the displacements of pixels between two image frames) as a type of mid-level representation (e.g., [1]}, [2]}) could deliver misleading information to the control module. We argue that both flow components are necessary and would contribute to the learning process of the control module, and thus should be separately treated. The second objective is to investigate the interplay of factorized flows with the other types of mid-level representations, and examine the impacts of different compositions of them on the control module's performance.
