The matching of the soundtrack in a movie or a video can have an enormous influence in the message being con- veyed and its impact, in the sense of involvement and engagement, and ultimately in their aesthetic and entertainment qualities. Art is often associated with creativity, implying the presence of inspiration, originality and appropriateness. Evolutionary systems provides us with the novelty, showing us new and subtly different solutions in every generation, possibly stimulating the creativity of the human using the system. In this paper, we present Genetic Soundtracks, an evolutionary approach to the creative matching of audio to a video. It analyzes both media to extract features based on their content, and adopts genetic algorithms, with the purpose of truncating, combining and adjusting audio clips, to align and match them with the video scenes. Index Terms— Genetic algorithms, multimedia, entertainment, feature extraction, audio & video signal processing, video editing
We present methods for automatic and semi-automatic creation of music videos, given an arbitrary audio soundtrack and source video. Significant audio changes are automatically detected; similarly, the source video is automatically segmented and analyzed for suitability based on camera motion and exposure. Video with excessive camera motion or poor contrast is penalized with a high unsuitability score, and is more likely to be discarded in the final edit. High quality video clips are then automatically selected and aligned in time with significant audio changes. Video clips are adjusted to match the audio segments by selecting the most suitable region of the desired length. Besides a fully automated solution, our system can also start with clips manually selected and ordered using a graphical interface. The video is then created by truncating the selected clips (preserving the high quality portions) to produce a video digest that is synchronized with the soundtrack music, thus enhancing the impact of both. In this paper we present Synesthetic Video, an interactive video that allows to experience video in cross-sensorial ways, to hear its colors and to influence its visual properties with sound and music, through user interaction or ambient influence. Our main motivations include accessibility, enriching users experiences, stimulating and supporting users creativity, and to learn more about synesthesia and how videos can influence and be influenced by users and the ambient, at the crossroads of art, science and technology.