After calculating the soft assignments for both views, we must align the two sets of \(k\)  clusters, since the \(j\) -th image cluster does not necessarily correspond to the \(j\) -th text cluster. To achieve this, we use the popular Hungarian algorithm [1]} to obtain the alignment between image clusters and text clusters. Hungarian algorithm is an optimization algorithm to solve the assignment problem by minimizing the assignment cost. We create a \(k \times k\)  confusion matrix where an entry \((m, n)\)  represents the number of data points being assigned to \(m\) -th image cluster and \(n\) -th text cluster. We then subtract the maximum value of the matrix from the value of each cell to obtain the “cost.“ The Hungarian algorithm is then applied to the cost matrix to find a clustering assignment with the minimum cost.
