Similar to our image-text-sound joint embedder, several works [1]}, [2]}, [3]} have attempted to extend the CLIP model to jointly represent audio, text, and images. Here, we also compare those works with our learned embedding space to analyze their effectiveness for the image manipulation task. As shown in Fig. REF , we qualitatively compare image manipulation results based on different multi-modal joint embedding spaces learned by Wav2CLIP [1]}, AudioCLIP [2]}, Lee et al. [3]}, and our joint embedder. Given different sound inputs (e.g., bird song, bomb, weak thunder, etc.), our learned embedding space generally produces the best results compared with the other embedders (see 2nd, 3rd, and 4th rows vs. 5th row). This may confirm that our multi-modal joint embedding space is more effective for the image manipulation task than are existing approaches.
<FIGURE>