A compositional approach to analysis re-synthesis
This report summarises the work achieved in the framework of a composer in research project in the IMTR team at IRCAM. In this project we have experimented with different realtime processing techniques for the analysis, transformation, and resynthesis of particular morphological aspects of live performed voice and instrumental sounds. Based on segment-wise sound descriptions, the developed tools allow for re-orchestrating and re-organising particular morphological aspects of the live performer's sound using concatenative synthesis of pre-analysed sound materials driven by the real-time analysis. The relationship between the morphology of live performance and the synthesised sound materials is controlled by a set of compositional tools manipulating the extracted sound descriptions in real-time.
First Steps in Relaxed Real-Time Typo-Morphological Audio Analysis/Synthesis
This paper describes a real-time audio analysis/resynthesis system that we developed for a music piece for ensemble and electronics. The system combines real-time audio analysis and concatenative synthesis based on the segmentation of sound streams into constituting segments and the description of segments by an efficient set of descriptors adapted to the given musical context. The system has been implemented in Max/MSP using the FTM & Co and MuBu libraries and successfully employed in the production and performance of the piece. As more and more research in the domain of music information retrieval, we use the term of typo-morhpology to designate the description of sounds by morphologic criteria including the temporal evolution of sound features that also can provide pertinent means for the classification of sounds. Although, the article mainly insists on the technical aspects of the work, it occasionally contextualizes the different technical choices regarding particular musical aspects.
Sound Indexing Using Morphological Description
Sound sample indexing usually deals with the recognition of the source/cause that has produced the sound. For abstract sounds, sound effects, unnatural, or synthetic sounds, this cause is usually unknown or unrecognizable. An efficient description of these sounds has been proposed by Schaeffer under the name morphological description. Part of this description consists in describing a sound by identifying the temporal evolution of its acoustic properties to a set of profiles. In this paper, we consider three morphological descriptions: dynamic profiles (ascending, descending, ascending/descending, stable, impulsive), melodic profiles (up, down, stable, up/down, down/up) and complex-iterative sound description (non-iterative, iterative, grain, repetition). We study the automatic indexing of a sound into these profiles. Because this automatic indexing is difficult using standard audio features, we propose new audio features to perform this task. The dynamic profiles are estimated by modeling the loudness over-time of a sound by a second-order B-spline model and derive features from this model. The melodic profiles are estimated by tracking over time the perceptual filter which has the maximum excitation. A function is derived from this track which is then modeled using a second-order B-spline model. The features are again derived from the B-spline model. The description of complex-iterative sounds is obtained by estimating the amount of repetition and the period of the repetition. These are obtained by computing an audio similarity function derived from an Mel frequency cepstral coefficients (MFCC) similarity matrix. The proposed audio features are then tested for automatic classification. We consider three classification tasks corresponding to the three profiles. In each case, the results are compared with the ones obtained using standard audio features.