To quantify our model's degree of disentanglement, we refer to the classic DCI and modularity metrics [1]}, [2]}, which measure the extant to which there is 1-to-1 correspondence between latent factors and image attributes. Following the protocol in [3]}, [4]}, we consider a sample set of synthesized CLEVR images, using a pre-trained object detector to extract and summarize their visual and semantic attributes. Note that since the CLEVR object detector reaches accuracy of 0.997, it does not introduce imprecisions into the evaluation. To further assess the level of controllability, we perform random perturbations over the latent variables used to generate the scenes, and estimate the mean correlation between the resulting semantic changes among object and property pairs. Intuitively, we expect a disentangled representation to yield uncorrelated changes among objects and attributes. As shown in table REF , GANformer2 attains higher disentanglement and stronger controllability than leading GAN-based and variational approaches, quantitatively confirming the efficacy of our approach.
<FIGURE>