After learning the multi-modal joint embedding space by minimizing Eq. REF , we use a direct latent code optimization method, similar to StyleCLIP [1]}, to manipulate the given image. Our method optimizes for each image instance manipulated at the inference stage. Our model minimizes the distance between a given source latent code and an audio-driven latent code in the learned joint embedding space to produce sound-guided manipulated images. Moreover, we propose an Adaptive Layer Masking technique, which adaptively manipulates the latent code.
