Self Paced Cross Modal Subspace Matching
Cross modal matching methods match data from different modalities according to their similarities. Most existing methods utilize label information to reduce the semantic gap between different modalities. However, it is usually time consuming to manually label large scale data. This paper proposes a Self Paced Cross Modal Subspace Matching (SCSM) method for unsupervised multimodal data. We assume that multimodal data are pair wised and from several semantic groups, which form hard pair wised constraints and soft semantic group constraints respectively. Then, we formulate the unsupervised cross modal matching problem as a non convex joint feature learning and data grouping problem. Self paced learning, which learns samples from 'easy' to 'complex' is further introduced to refine the grouping result. Moreover, a multimodal graph is constructed to preserve the relationship of both inter and intra modality similarity. An alternating minimization method is employed to minimize the non convex optimization problem, followed by the discussion on its convergence analysis and computational complexity. Experimental results on four multimodal databases show that SCSM outperforms state of the art cross modal subspace learning methods.
SIGIR