Evaluation metrics.
To quantitatively evaluate our experiments we adopt a set of evaluation metrics widely used in the disentanglement literature: namely the \(\beta \) -VAE score and the Factor VAE score, which measure disentanglement as the accuracy of a linear classifier that predicts the index of a fixed factor of variation and a majority vote classifier respectively [1]}, [2]}; the DCI disentanglement score, which measures the entropy of the distribution on the single dimension of the representation vector [3]}; and the mutual information gap (MIG) [4]} which for each dimension of the representation vector, first measures the mutual information w.r.t. the other dimensions and then takes the gap between the highest and second coordinate.
Note that the aforementioned metrics have been defined to measure the disentanglement in a variational framework, wherein the factors of variation are assumed to be one-dimensional. This is not the case in our method, where each disentangled subspace is not required to be one-dimensional.
Nevertheless, we can still apply these metrics in the aggregated latent space. This allows us to perform a direct comparison with methods working in the variational setting.
