To learn the multi-modal joint embedding space, we use the pre-trained CLIP [1]}-based image and text encoders, and we train a Swin transformer [2]}-based audio encoder, which outputs a latent vector of the length (i.e., 512) that is the same as the output of the image and text encoders. Note that we convert audio inputs into Mel-spectrogram acoustic features by following a common practice in various audio processing tasks. For the image manipulation step, we leverage the pre-trained StyleGAN2 [3]} generator. Note that we adjust the size of the latent code depending on the resolution of the output image, i.e., \(18 \times 512\)  latent code for images of size \(1024 \times 1024\)  and \(14 \times 512\)  for \(256 \times 256\) . To implement such a generator, we utilize the official PyTorch implementation from StyleGAN2-ADAhttps://github.com/NVlabs/stylegan2-ada-pytorch.
