The proposed approach for unsupervised optical flow-segmentation is based on a generative-adversarial approach, constrained by a cycle-consistency loss. This approach allows to learn the mapping between the domain of optical-flow images and the domain of shape-priors, consisting of realistic binary segmentation masks of the target object (in this case surgical tools), without requiring pairwise matching between the two domains.
The method is inspired by the classic cycle-GAN architecture ([1]}), a popular generative architecture for image-to-image translation from unpaired domains. However, it is known that mapping between a domain of minimal complexity, as the binary shape-priors, lacking of strong discriminative features, and a more complex one, such as the optical-flow, is an ill-posed problem, suffering from issues such as information-hiding (`steganography' [2]}) and overpowering discriminator, possibly hindering the whole training process.
