We propose a method to manipulate images based on the semantic-level understanding from the given audio input. We take the user-provided audio input into the latent space of StyleGAN2 [1]} and the CLIP [2]} embedding space. In particular, the strategy of adding a semantic edge between audio-visual from text obtains a more robust sound representation for fine-features. Then, the latent code is aligned with the audio to enable meaningful image manipulation while reflecting the context from the audio. Our model produces responsive manipulations based on various audio inputs. We observe that an audio input can successfully provide a semantic cue to manipulate images accordingly. Our method of traversing multi-modal embedding space can be used in many applications with multi-modal contexts.
<FIGURE>