Image manipulation is a task that naturally edits images according to the user's intention. Recent studies provide guidance on manipulation using multi-modal information such as sketches [1]}, text [2]}, [3]}, [4]}, and sound [5]}. To apply the user's intention into the image, a mixture of sketches and text is used to perform image manipulation and synthesis [1]}, [2]}. The user's intention can be applied by drawing a picture [1]} or writing text with semantic meanings [2]}, [4]}. We mainly focus on image modification based on user-provided sound. Because discrete input like text cannot fully express the user's intention when it is used alone, sound provides an alternative option, as it provides other properties such as continuity and dynamics over time. Such properties of sound allow image manipulation to be more dynamic and create different levels of semantics or sentiment in a scene. For example, every video clip of “thunder” generates a different loudness and different characteristics of “sound of thunder”. In addition, when a piece of music is expressed with a combination of several instruments, the atmosphere of the music can be expressed only with sound.
<FIGURE>