US8290167B2 - Method and apparatus for conversion between multi-channel audio formats - Google Patents
Method and apparatus for conversion between multi-channel audio formats Download PDF
audio signal
Ville Pulkki
Juergen Herre
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
2007-04-30 Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
2007-07-24 Assigned to FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V. reassignment FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: PULKKI, VILLE, HERRE, JUERGEN
238000006243 chemical reaction Methods 0 claims description title 27
238000004458 analytical methods Methods 0 description 15
238000004091 panning Methods 0 description 3
238000002156 mixing Methods 0 description 2
238000001308 synthesis method Methods 0 description 1
H04S—STEREOPHONIC SYSTEMS
H04S3/00—Systems employing more than two channels, e.g. quadraphonic
H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
G10—MUSICAL INSTRUMENTS; ACOUSTICS
G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
G10L19/16—Vocoder architecture
G10L19/173—Transcoding, i.e. converting between two coded representations avoiding cascaded coding-decoding
G10L19/008—Multichannel audio signal coding or decoding, i.e. using interchannel correlation to reduce redundancies, e.g. joint-stereo, intensity-coding, matrixing
H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
H04S2420/11—Application of ambisonics in stereophonic audio systems
An input multi-channel representation is converted into a different output multi-channel representation of a spatial audio signal, in that an intermediate representation of the spatial audio signal is derived, the intermediate representation having direction parameters indicating a direction of origin of a portion of the spatial audio signal; and in that the output multi-channel representation of the spatial audio signal is generated using the intermediate representation of the spatial audio signal.
The present invention relates to a technique as to how to convert between different multi-channel audio formats in the highest possible quality without being limited to specific multi-channel representations. That is, the present invention relates to a technique allowing the conversion between arbitrary multi-channel formats.
BACKGROUND OF THE INVENTION AND PRIOR ART
Generally, in multi-channel reproduction and listening, a listener is surrounded by multiple loudspeakers. Various methods exist to capture audio signals for specific setups. One general goal in the reproduction is to reproduce the spatial composition of the originally recorded sound event, i.e. the origins of individual audio sources, such as the location of a trumpet within an orchestra. Several loudspeaker setups are fairly common and can create different spatial impressions. Without using special post-production techniques, the commonly known two-channel stereo setups can only recreate auditory events on a line between the two loudspeakers. This is mainly achieved by so-called "amplitude-panning", where the amplitude of the signal associated to one audio source is distributed between the two loudspeakers, depending on the position of the audio source with respect to the loudspeakers. This is normally done during recording or subsequent mixing. That is, an audio source coming from the far-left with respect to the listening position will be mainly reproduced by the left loudspeaker, whereas an audio source in front of the listening position will be reproduced with identical amplitude (level) by both loudspeakers. However, sound emanating from other directions cannot be reproduced.
Consequently, by using more loudspeakers that are distributed around the listener, more directions can be covered and a more natural spatial impression can be created. The probably most well known multi-channel loudspeaker layout is the 5.1 standard (ITU-R775-1), which consists of 5 loudspeakers, whose azimuthal angles with respect to the listening position are predetermined to be 0°, ±30° and ±110°. That means, during recording or mixing, the signal is tailored to that specific loudspeaker configuration and deviations of a reproduction setup from the standard will result in decreased reproduction quality.
Numerous other systems with varying numbers of loudspeakers located at different directions have also been proposed. Professional and special systems, especially in theaters and sound installations, do also include loudspeakers at different heights.
A universal audio reproduction system named DirAC has been recently proposed which is able to record and reproduce sound for arbitrary loudspeaker setups. The purpose of DirAC is to reproduce the spatial impression of an existing acoustical environment as precisely as possible, using a multi-channel loudspeaker system having an arbitrary geometrical setup. Within the recording environment, the responses of the environment (which may be continuous recorded sound or impulse responses) are measured with an omnidirectional microphone (W) and with a set of microphones allowing to measure the direction of arrival of sound and the diffuseness of sound. In the following paragraphs and within the application, the term "diffuseness" is to be understood as a measure for the non-directivity of sound. That is, sound arriving at the listening or recording position with equal strength from all directions, is maximally diffuse. A common way to quantify diffusion is to use diffuseness values from the interval [0, . . . , 1], wherein a value of 1 describes maximally diffuse sound and value of 0 describes perfectly directional sound, i.e. sound emanating from one clearly distinguishable direction only. One commonly known method of measuring the direction of arrival of sound is to apply 3 figure-of-eight microphones (XYZ) aligned with Cartesian coordinate axes. Special microphones, so-called "SoundField microphones", have been designed, which directly yield all the desired responses. However, as mentioned above, the W, X, Y and Z signals may also be computed from a set of discrete omnidirectional microphones.
Another method to store audio formats for arbitrary number of channels to one or two downmix channels of audio with accompanying directional data has been recently proposed by Goodwin and Jot. This format can be applied to arbitrary reproduction systems. The directional data, i.e. the data having information about the direction of audio sources is computed using "Gerzon vectors", which consist of a velocity vector and an energy vector. The velocity vector is a weighted sum of vectors pointing at loudspeakers from the listening position, wherein each weight is the magnitude of a frequency spectrum at a given time/frequency tile for a loudspeaker. The energy vector is a similarly weighted vector sum. However, the weights are short-time energy estimates of the loudspeaker signals, that is, they describe a somewhat smoothed signal or an integral of the signal energy contained in the signal within finite length time-intervals. These vectors share the disadvantage of not being related to a physical or a perceptual quantity in a well-grounded way. For example, the relative phase of the loudspeakers with respect to each other is not properly taken into account. That means, for example, if a broadband signal is fed into the loudspeakers of a stereophonic setup in front of a listening position with opposite phase, a listener would perceive sound from ambient direction, and the sound field in the listening position would have sound energy oscillations from side to side (e.g. from the left side to the right side). In such a scenario, the Gerzon vectors would be pointing towards the front direction, which is obviously not representing the physical or the perceptual situation.
Naturally, having multiple multi-channel formats or representations in the market, the requirement exists to be able to convert between the different representations, such that the individual representations may be reproduced with setups originally developed for the reconstruction of an alternative multi-channel representation. That is, for example, a transformation between the 5.1 channels and 7.1 or 7.2 channels may be required to use an existing 7.1 or 7.2 channel playback setup for playing back the 5.1 multi-channel representation commonly used on DVD. The great variety of audio formats makes the audio content production difficult, as all formats require specific mixes and storage/transmission formats. Therefore, conversion between different recording formats for playback on different reproduction setups is necessary.
There are a number of methods proposed to convert audio in a specific audio format to another audio format. However, these methods are always tailored to specific multi-channel formats or representations. That is, these are only applicable to the conversion from one specific predetermined multi-channel representation into another specific multi-channel representation.
Generally, a reduction in the number of reproduction channels (so-called "downmix") is simpler to implement that an increase in the number of reproduction channels ("upmix"). For some standard loudspeaker reproduction setups, recommendations are provided by, for example, the ITU on how to downmix to reproduction setups with a lower number of reproduction channels. In these so-called "ITU" downmix equations, the output signals are derived as simple static linear combinations of input signals. Usually, a reduction of the number of reproduction channels leads to a degradation of the perceived spatial image, i.e. a degraded reproduction quality of a spatial audio signal.
For a possible benefit from a high number of reproduction channels or reproduction loudspeakers, upmixing techniques for specific types of conversions have been developed. An often investigated problem is how to convert 2-channel stereophonic audio for reproduction with 5-channel surround loudspeaker systems. One approach or implementation to such a 2-to-5 upmix is to use a so-called "matrix" decoder. Such decoders have become common to provide or upmix 5.1 multi-channel sound over stereo transmission infrastructures, especially in the early days of surround sound for movies and home theatres. The basic idea is to reproduce sound components which are in-phase in the stereo signal in the front of the sound image, and to put out-of-phase components into the rear loudspeakers. An alternative 2-to-5 upmixing method proposes to extract the ambient components of the stereo signal and to reproduce those components via the rear loudspeakers of the 5.1 setup. An approach following the same basic ideas on a perceptually more justified basis and using a mathematically more elegant implementation has been recently proposed by C. Faller in "Parametric Multi-channel Audio Coding: Synthesis of Coherence Cues", IEEE Trans. On Speech and Audio Proc., vol. 14, no. 1, Jan. 2006.
The recently published standard MPEG surround performs an upmix from one or two downmixed and transmitted channels to the final channels used in reproduction or playback, which is usually 5.1. This is implemented either using spatial side information (side information similar to the BCC technique) or without side information, by using the phase relations between the two channels of a stereo downmix ("non-guided mode" or "enhanced matrix mode").
All methods for format conversion described in the previous paragraphs are specialized to be applied to specific configurations of both the source and the destination audio reproduction format and are thus not universal. That is, a conversion between arbitrary input multi-channel representations to arbitrary output multi-channel representations cannot be performed. That is to say the prior art transformation techniques are specifically tailored to the number of loudspeakers and their precise position for the input multi-channel audio representation as well as for the output multi-channel representation.
It is, naturally, desirable to have a concept for multi-channel transformation which is applicable to arbitrary combinations of input and output multi-channel representations.
According to one embodiment of the present invention, an apparatus for conversion of an input multi-channel representation into a different output multi-channel representation of a spatial audio signal comprises: an analyzer for deriving an intermediate representation of the spatial audio signal, the intermediate representation having direction parameters indicating a direction of origin of a portion of the spatial audio signal; and a signal composer for generating the output multi-channel representation of the spatial audio signal using the intermediate representation of the spatial audio signal.
In that an intermediate representation is used which has direction parameters indicating a direction of origin of a portion of the spatial audio signal, conversion can be achieved between arbitrary multi-channel representations, as long as the loudspeaker configuration of the output multi-channel representation is known. It is important to note that the loudspeaker configuration of the output multi-channel representation does not have to be known in advance, that is, during the design of the conversion apparatus. As the conversion apparatus and method are universal, a multi-channel representation provided as an input multi-channel representation and designed for a specific loudspeaker-setup may be altered on the receiving side, to fit the available reproduction setup such that the reproduction quality of a reproduction of a spatial audio signal is enhanced.
According to a further embodiment of the present invention, the direction of origin of a portion of the spatial audio signal is analyzed within different frequency bands. Such, different direction parameters are derived for finite with frequency portions of the spatial audio signal. To derive the finite width frequency portions, a filterbank or a Fourier-transform may, for example, be used. According to another embodiment, the frequency portions or frequency bands, for which the analysis is performed individually is chosen to match the frequency resolution of the human hearing process. These embodiments may have the advantage that the direction of origin of portions of the spatial audio signal is performed as good as the human auditory system itself can determine the direction of origin of audio signals. Therefore, the analysis is performed without a potential loss of precision in the determination of the origin of an audio object or a signal portion, when a such analyzed signal is reconstructed and played back via an arbitrary loudspeaker setup.
According to a further embodiment of the present invention, one or more downmix channels are additionally derived belonging to the intermediate representation. That is, downmixed channels are derived from audio channels corresponding to loudspeakers associated to the input multi-channel representation, which may then be used for generating the output multi-channel representation or for generating audio channels corresponding to loudspeakers associated to the output multi-channel representation.
For example, a monophonic downmix a channel may be generated from the 5.1 input channels of a common 5.1 channel audio signal. This could, for example, be performed by computing the sum of all the individual audio channels. Based on the such derived monophonic downmix channel, a signal composer may distribute such portions of the monophonic downmix channel corresponding to the analyzed portions of the input multi-channel representation to the channels of the output multi-channel representation as indicated by the direction parameters. That is, a frequency/time or signal portion analyzed to be coming from the far left from a spatial audio signal will be redistributed to the loudspeakers of the output multi-channel representation, which are located on the left side with respect to a listening position.
Generally, some embodiments of the present invention allow to distribute portions of the spatial audio signal with greater intensity to a channel corresponding to a loudspeaker closer to the direction indicated by the direction parameters than to a channel further away from that direction. That is, no matter how the location of loudspeakers used for reproduction are defined in the output multi-channel representation, a spatial redistribution will be achieved fitting the available reproduction setup as good as possible.
According to some embodiments of the present invention, a spatial resolution, with which a direction of origin of a portion of the spatial audio signal can be determined, is much higher than the angle of three dimensional space associated to one single loudspeaker of the input multi-channel representation. That is, the direction of origin of a portion of the spatial audio signal can be derived with a better precision than a spatial resolution achievable by simply redistributing the audio channels from one distinct setup to another specific setup, as for example by redistributing the channels of a 5.1 setup to a 7.1 or 7.2 setup.
Summarizing, some embodiments of the invention allow the application of an enhanced method for format conversion which is universally applicable and does not depend on a particular desired target loudspeaker layout/configuration. Some embodiments convert an input multi-channel audio format (representation) with N1 channels into an output multi-channel format (representation) having N2 channels by means of extracting direction parameters (similar to DirAC), which are then used for synthesizing the output signal having N2 channels. Furthermore, according to some embodiments, a number of N0 downmix channels are computed from the N1 input signals (audio channels corresponding to loudspeakers according to the input multi-channel representation), which are then used as a basis for a decoding process using the extracted direction parameters.
Several embodiments of the present invention will in the following be described referencing the enclosed drawings.
FIG. 1 shows an illustration of derivation of direction parameters indicating a direction of origin of a portion of an audio signal; and
FIG. 2 shows a further embodiment of derivation of direction parameters based on a 5.1-channel representation;
FIG. 3 shows an example of generation of an output multi-channel representation;
FIG. 4 shows an example for audio conversion from a 5.1-channel setup to an 8.1 channel setup; and
FIG. 5 shows an example for an inventive apparatus for conversion between multi-channel audio formats.
Some embodiments of the present invention derive an intermediate representation of a spatial audio signal having direction parameters indicating a direction of origin of a portion of the spatial audio signal. One possibility is to derive a velocity vector indicating the direction of origin of a portion of a spatial audio signal. One example for doing so will be described in the following paragraphs, referencing FIG. 1.
Before detailing the concept, it may be noted that the following analysis may be applied to multiple individual frequency or time portions of the underlying spatial audio signal simultaneously. For the sake of simplicity, however, the analysis will be described for one specific frequency or time or time/frequency portion only. The analysis is based on an energetic analysis of the sound field recorded at a recording position 2, located at the center of a coordinate system, as indicated in FIG. 1.
The coordinate system is a Cartesian Coordinate System, having an x axis 4 and a y axis 6 perpendicular to each other. Using a right handed system, the z axis not shown in FIG. 1 points to the direction out of the drawing plane.
For the direction analysis, it is assumed that 4 signals (known as B-format signals) are recorded. One omnidirectional signal w is recorded, i.e. a signal receiving signals from all directions with (ideally) equal sensitivity. Furthermore, three directional signals X, Y and Z are recorded, having a sensitivity distribution pointing in the direction of the axes of the Cartesian Coordinate System. Examples for possible sensitivity patterns of the microphones used are given in FIG. 1 showing two "figure-of-eight" patterns 8 a and 8 b, pointing to the directions of the axes. Two possible audio sources 10 and 12 are furthermore illustrated in the two-dimensional projection of the coordinate system shown in FIG. 1.
For the direction analysis, an instantaneous velocity vector (at time index n) is composed for different frequency portions (described by the index i) by
v(n,i)=X(n,i)e x +Y(n,i)e y +Z(n,i)e z. (1)
That is, a vector is created having the individually recorded microphone signals of the microphones associated to the axis of the coordinate system as components. In the previous and the following equations, the Quantities are indexed in Time (n) as well as in frequency (i) by two indices (n, i). That is,
ex, ey and ez represent Cartesian unit vectors.
Using the simultaneously recorded omnidirectional signal w, an instantaneous intensity I is computed as
I(n,i)=w(n,i)v(n,i), (2)
the instantaneous energy is derived according to the following formula:
E(n,i)=w 2(n,i)+∥v∥ 2(n,i), (3)
where ∥ ∥ denotes vector norm.
That is, an intensity quantity is derived allowing for possible interference between two signals (as positive and negative amplitudes may occur). Additionally, an energy quantity is derived, which naturally does not allow for interference between two signals, as the energy quantity does not contain negative values allowing for an cancellation of the signal.
These properties of the intensity and the energy signals can be advantageously used to derive a direction of origin of signal portions with high accuracy, preserving a virtual correlation of audio channels (a relative phase between the channels), as it will be detailed below.
On the one hand, the instantaneous intensity vector may be used as vector indicating the direction of origin of a portion of the spatial audio signal. However, this vector may undergo rapid changes thus causing artifacts within the reproduction of the signal. Therefore, alternatively, an instantaneous direction may be computed using short time averaging utilizing a Hanning window W2 according to the following formula:
D ⁡ ( n , i ) = - ∑ m = - M / 2 M / 2 ⁢ I ⁡ ( n + m , i ) ⁢ W 2 ⁡ ( m ) , ( 4 )
where W2 is the Hanning window for short-time averaging D.
That is, optionally, a short-time averaged direction vector having parameters indicating a direction of origin of the spatial audio signal may be derived.
Optionally, a diffuseness measure ψ may be computed as follows:
ψ ⁡ ( n , i ) = 1 - ∑ m = - M / 2 M / 2 ⁢  I ( n + m , i  2 ⁢ W 1 ⁡ ( m ) ∑ m = - M / 2 M / 2 ⁢ E ⁡ ( n + m , i ) ⁢ W 1 ⁡ ( m ) ( 5 )
where W1(m) is a window function defined between −M/2 and M/2 for short-time averaging.
It should again be noted that the deriving is performed such as to preserve virtual correlation of the audio channels. That is, phase information is properly taken into account, which is not the case for direction estimates based on energy estimates only (as for example Gerzon vectors).
The following simple example shall serve to explain this in more detail. Consider a perfectly diffuse signal which is played back by two loudspeakers of a stereo system. As the signal is diffuse (originating from all directions), it is to be played back by both speakers with equal intensity. However, as the perception shall be diffuse, a phase shift of 180 degrees is required. In such a scenario, a purely energy based direction estimation would yield a direction vector pointing exactly to the middle between the two loudspeakers, which certainly is a undesirable result not reflecting reality.
According to the inventive concept detailed above, virtual correlation of the audio channels is preserved while estimating the direction parameters (direction vectors). In this particular example, the direction vector would be zero, indicating that the sound does not originate from one distinct direction, which is clearly not the case in reality. Correspondingly, the diffuseness parameter of equation (5) is 1, matching the real situation perfectly.
The Hanning windows in the above equations may furthermore have different lengths for different frequency bands.
As a result of this analysis, for each time slice of a frequency portion, a direction vector or direction parameters are derived indicating a direction of origin of the portion of the spatial audio signal, for which the analysis has been performed. Optionally, a diffuseness parameter can be derived indicating the diffuseness of the direction of a portion of the spatial audio signal. As previously described, a diffusion value of one derived according to equation (4) describes a signal of maximal diffuseness, i.e. originating from all directions with equal intensity.
To the contrary, small diffuseness values are attributed to signal portions originating predominantly from one direction.
FIG. 2 shows an example for the derivation of direction parameters from an input multi-channel representation having five channels according to ITU-775-1. The multi-channel input audio signal, i.e. the input multi-channel representation, is first transformed into B-format by simulating an anechoic recording of the corresponding multi-channel audio setup. With respect to a center 20 of the Cartesian Coordinate System having an axis x 22 and y 24, a rear-right loudspeaker 26 is located at an angle of 110°. A right-front loudspeaker 28 is located at +30°, a center loudspeaker at 0°, a left-front loudspeaker 32 at −31° and a left-rear loudspeaker 34 at −110°. In practice, an anechoic recording can be simulated by applying simple matrixing operations, the geometrical setup of the input multi-channel representation is known.
An omnidirectional signal w can be obtained by taking a direct sum of all loudspeaker signals, that is of all audio channels corresponding to the loudspeakers associated to the input multi-channel representation. The dipole or "figure-of-eight" signals X, Y and Z can be formed by adding the loudspeaker signals weighted by the cosine of the angle between the loudspeaker and the corresponding Cartesian axes, i.e. the direction of maximum sensitivity of the dipole microphone to be simulated. Let Ln be the 2-D or 3-D Cartesian vector pointing towards the nth loudspeaker and V be the unit vector pointing to the Cartesian axis direction corresponding to the dipole microphone. Then, the weighting factor is cos(angle(Ln, V)). The directional signal X would, for example, be written as
X = ∑ n = 1 N ⁢ C n · cos ⁡ ( angle ⁡ ( L n , V ) ) ,
when Cn denotes the loudspeaker signal of the nth channel and N is the number of channels. The term angle has to be interpreted as an operator, computing the spatial angle between the two given vectors. That is, for example the angle 40 (Θ) between the Y axis 24 and the left-front loudspeaker 32 in the two dimensional case illustrated in FIG. 2.
The further derivation of direction parameters could, for example, be performed as illustrated in FIG. 1 and detailed in the corresponding description, i.e. audio signals X, Y and Z can be divided into frequency bands according to frequency resolution of the human auditory system. The direction of the sound, i.e. the direction of origin of the portions of the spatial audio signal and, optionally, diffuseness is analyzed depending on time in each frequency channel. Optionally, a replacement for sound diffuseness using another measure of signal dissimilarity than diffuseness can also be used, such as the coherence between (stereo) channels associated to the spatial audio signal.
If, as a simplified example, one audio source 44 is present, as indicated in FIG. 2, wherein that source only contributes to the signal within a specific frequency band, a direction vector 46 pointing to the audio source 44 would be derived. The direction vector is represented by direction parameters (vector components) indicating the direction of the portion of the spatial audio signal originating from audio source 44. In the reproduction setup of FIG. 2, such a signal would be reproduced mainly by the left-front loudspeaker 32 as illustrated by the symbolic wave form associated to this loudspeaker. However, minor signal portions will also be played back from the left-rear loudspeaker 32. Hence, the directional signal of the microphone associated to the X coordinate 22 would receive signal components from the left-front channel 32 (the audio channel associate to the left-front loudspeaker 32) and the left-rear channel 34.
As, according to the above implementation, the directional signal Y associated to the y-axis will receive also signal portions played back by the left-front loudspeaker 32, a directional analysis based on directional signals X and Y will be able to reconstruct sound coming from direction vector 46 with high precision.
For the final conversion to the desired multi-channel representation (multi-channel format), the direction parameters indicating the direction of origin of portions of the audio signals are used. Optionally, one or more (N0) additional audio downmix channels may be used. Such a downmix channel may, for example, be the omnidirectional channel W or any other monophonic channel. However, for the spatial distribution, the use of only one single channel associated to the intermediate representation is of minor negative impact. That is, several downmix channels, such as a stereo mix, the channels W, X and Y or all channels of a B-format may be used as long as the direction parameters or the directional data has been derived and can be used for the reconstruction or the generation of the output multi-channel representation. It is alternatively also possible to use the 5 channels of FIG. 2 directly or any combination of channels associated to the input multi-channel representation as replacement for possible downmix channels. When only one channel is stored, there might be a degradation of the quality in the reproduction of diffuse sound.
FIG. 3 shows an example for the reproduction of the signal of audio source 44 with a loudspeaker-setup differing significantly from the loudspeaker-setup of FIG. 2, which was the input multi-channel representation from which the parameters have been derived. FIG. 3 shows, as an example, six loudspeakers 50 a to 50 f equally distributed along a line in front of a listening position 60, defining the center of a coordinate system having an x-axis 22 and a y-axis 24, as introduced in FIG. 2. As a previous analysis has provided direction parameters describing the direction of the direction vector 46 pointing to the source of the audio signal 44, an output multi-channel representation adapted to the loudspeaker setup of FIG. 3 can easily be derived by redistributing the portion of the spatial audio signal to be reproduced to the loudspeakers close to the direction of audio source 44, i.e. by those loudspeakers close to the direction indicated by the direction parameters. That is, audio channels corresponding to loudspeakers in the direction indicated by the direction parameters are emphasized with respect to audio channels corresponding to loudspeakers far away from this direction. That is, loudspeakers 50 a and 50 b can be steered (for example using amplitude panning) to reproduce the signal portion, whereas loudspeakers 50 c to 50 f do not reproduce that specific signal portion, while they may be used for reproduction of diffuse sound or other signal portions of different frequency bands.
The use of a signal composer for generating the output multi-channel representation of the spatial audio signal using the direction parameters can also be interpreted as being a decoding of the intermediate signal into the desired multi-channel output format having N2 output channels. Audio downmix channels or signals generated are typically processed in the same frequency band as they have been analyzed in. Decoding may be performed in a manner similar to DirAC. In the optional reproduction of diffuse sound, the audio use for representing a non-diffuse stream is typically either one of the optional N0 downmix channel signals or linear combinations thereof.
For the optional creation of a diffuse stream, several synthesis options exist to create the diffuse part of the output signals or the output channels corresponding to loudspeakers according to the output multi-channel representation. If there is only one downmix channel transmitted, that channel has to be used to create non-diffuse signals for each loudspeaker. If there are more channels transmitted, there are more options how diffuse sound may be created. If, for example, a stereo downmix is used in the conversion process, an obviously suited method is to apply the left downmix channel to the loudspeakers on the left and the right downmix channel to the loudspeakers on the right side. If several downmix channels are used for the conversion (i.e. N0>1), the diffuse stream for each loudspeaker can be computed as a differently weighted sum of these downmix channels. One possibility could, for example, be transmitting a B-format signal (channels X, Y, Z and w as previously described) and computing the signal of a virtual cardioid microphone signal for each loudspeaker.
The following text describes a possible procedure for the conversion of an input multi-channel representation into an output multi-channel representation as a list. In this example, sound is recorded with a simulated B-format microphone and then further processed by a signal composer for listening or playing back with a multi-channel or a monophonic loudspeaker setup. The single steps are explained referencing FIG. 4 showing a conversion of a 5.1-channel input multi-channel representation into an 8-channel output multi-channel representation. The basis is a N1-channel audio format (N1 being 5 in the specific example). To convert the input multi-channel representation into a different output multi-channel representation the following steps may be performed.
1. Simulate an anechoic recording of an arbitrary multi-channel audio representation having N1 audio channels (5 channels), as illustrated in the recording section 70 (with a simulated B-format microphone in a center 72 of the layout).
2. In an analysis step 74, the simulated microphone signals are divided into frequency bands and in a directional analysis step 76, the direction of origin of portions of the simulated microphone signals are derived. Furthermore, optionally, diffuseness (or coherence) may be determined in a diffuseness termination step 78.
As previously mentioned a direction analysis may be performed without using a B-format intermediate step. That is, generally, an intermediate representation of the spatial audio signal has to be derived based on an input multi-channel representation, wherein the intermediate representation has direction parameters indicating a direction of origin of a portion of the spatial audio signal.
3. In a downmix step 80, N0 downmix audio signals are derived, to be used as the basis for the conversion/the creation of the output multi-channel representation. In a composition step 82, the N0 downmix audio signals are decoded or upmixed to an arbitrary loudspeaker setup requiring N2 audio channels by an appropriate synthesis method (for example using amplitude panning or equally suitable techniques).
The result can be reproduced by a multi-channel loudspeaker system, having for example 8 loudspeakers as indicated in the playback scenario 84 of FIG. 4. However, thanks to the universality of the concept, a conversion may also be performed to a monophonic loudspeaker setup, providing an effect as if the spatial audio signal had been recorded with one single directional microphone.
FIG. 5 shows a principle sketch of an example for an apparatus for conversion between multi-channel audio formats 100.
The Apparatus 100 for receives an input multi-channel representation 102.
The Apparatus 100 comprises an analyzer 104 for deriving an intermediate representation 106 of the spatial audio signal, the intermediate representation 106 having direction parameters indicating a direction of origin of a portion of the spatial audio signal.
The Apparatus 100 furthermore comprises a signal composer 108 for generating a output multi-channel representation 110 of the spatial audio signal using the intermediate representation (106) of the spatial audio signal.
Summarizing, the embodiments of the conversion apparatuses and conversion methods previously described provide some great advantages. First of all, virtually any input audio format can be processed in this way. Moreover, the conversion process can generate output for any loudspeaker layout, including non-standard loudspeaker layout/configurations without the need to specifically tailor new relations for new combinations of input loudspeaker layout/configurations and output loudspeaker layout/configurations. Furthermore, the spatial resolution of audio reproduction increases when the number of loudspeakers is increased, contrary to prior art implementations.
Depending on certain implementation requirements of the inventive methods, the inventive methods can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, in particular a disk, DVD or a CD having electronically readable control signals stored thereon, which cooperate with a programmable computer system such that the inventive methods are performed. Generally, the present invention is, therefore, a computer program product with a program code stored on a machine readable carrier, the program code being operative for performing the inventive methods when the computer program product runs on a computer. In other words, the inventive methods are, therefore, a computer program having a program code for performing at least one of the inventive methods when the computer program runs on a computer.
While the foregoing has been particularly shown and described with reference to particular embodiments thereof, it will be understood by those skilled in the art that various other changes in the form and details may be made without departing from the spirit and scope thereof. It is to be understood that various changes may be made in adapting to different embodiments without departing from the broader concepts disclosed herein and comprehended by the claims that follow.
1. Apparatus for conversion of an input multi-channel representation into a different output multi-channel representation of a spatial audio signal, comprising:
an analyzer configured for deriving an intermediate representation of the spatial audio signal, the intermediate representation comprising direction parameters indicating a direction of origin of a portion of the spatial audio signal and at least one downmix channel; and
a signal composer configured for generating the output multi-channel representation of the spatial audio signal using the intermediate representation of the spatial audio signal by performing an upmixing operation, wherein the at least one downmix channel and the direction parameters are used in the upmixing operation.
2. Apparatus in accordance with claim 1, in which the analyzer is operative to derive direction parameters depending on a virtual correlation of audio channels associated to the input multi-channel representation.
3. Apparatus in accordance with claim 1, in which the analyzer is operative to derive direction parameters preserving the relative phase information of audio channels associated to the input multi-channel representation.
4. Apparatus in accordance with claim 1, in which the analyzer is operative to derive different direction parameters for finite width frequency portions of the spatial audio signal.
5. Apparatus in accordance with claim 1, in which the analyzer is operative to derive different direction parameters for finite length time portions of the spatial audio signal.
6. Apparatus in accordance with claim 4, in which the analyzer is operative to derive the different direction parameters for finite length time portions of the spatial audio signal associated to the frequency portions, wherein the length of a first time portion associated to a first frequency portion differs from the length of a second time portion association to a second, different frequency portion of the spatial audio signal.
7. Apparatus in accordance with claim 1, in which the analyzer is operative to derive direction parameters describing a vector pointing to the direction of origin of the portion of the spatial audio signal.
8. Apparatus in accordance with claim 1, in which the analyzer is additionally operative to derive one or more audio channels associated to the intermediate representation.
9. Apparatus in accordance with claim 8, in which the analyzer is operative to derive audio channels corresponding to loudspeakers associated to the input multi-channel representation.
10. Apparatus in accordance with claim 8, in which the analyzer is operative to derive one downmix channel as sum of audio channels corresponding to loudspeakers associated to the input multi-channel representation.
11. Apparatus in accordance with claim 8, in which the analyzer is operative to derive at least one audio channel associated to the direction of an axis of a Cartesian Coordinate System.
12. Apparatus in accordance with claim 11, in which the analyzer is operative to derive the at least one audio channel building the weighted sum of audio channels corresponding to loudspeakers associated to the input multi-channel representation.
13. Apparatus in accordance with claim 11, in which the analyzer is operative such that the deriving of the at least one audio channel X associated to the direction V of an axis of the Cartesian Coordinate System can be described by a combination of n audio channels Cn corresponding to all n loudspeakers associated to the input multi-channel representation and directed in a direction Cn, according to the following formula:
X = ∑ n = 1 N ⁢ C n · cos ⁡ ( angle ⁡ ( L n , V ) ) .
14. Apparatus in accordance with claim 1, in which the analyzer is further operative to derive a diffuseness parameter indicating a diffuseness of the direction of origin of the portion of the spatial audio signal.
15. Apparatus in accordance with claim 1, in which the signal composer is operative to distribute the portion of the spatial audio signal to a number of channels corresponding to a number of loudspeakers associated to the output multi-channel representation.
16. Apparatus in accordance with claim 15, in which the signal composer is operative such that the portion of the spatial audio signal is distributed with greater intensity to a channel corresponding to a loudspeaker closer to the direction indicated by the direction parameters than to a channel corresponding to a loudspeaker further away from that direction.
17. Apparatus in accordance with claim 14, in which the signal composer is operative such that the portion of the spatial audio signal is distributed with more uniform intensity to channels corresponding to loudspeakers associated to the output multi-channel representation when the diffuseness parameter indicates higher diffuseness than when the diffuseness parameter indicates lower diffuseness.
18. Apparatus in accordance with claim 1 further comprising:
an input interface for receiving the input multi-channel representation.
an input representation decoder for deriving a number of audio channels corresponding to all loudspeakers associated to the input multi-channel representation.
20. Apparatus in accordance with claim 15, in which the signal composer further comprises an output channel encoder for deriving the output multi-channel representation based on the audio channels corresponding to the loudspeakers associated to the output channel representation.
21. Apparatus in accordance with claim 1 further comprising an output interface for providing the output multi-channel representation.
22. Method for conversion of an input multi-channel representation into a different output multi-channel representation of a spatial audio signal, the method comprising:
deriving an intermediate representation of the spatial audio signal, the intermediate representation comprising direction parameters indicating a direction of origin of a portion of the spatial audio signal and at least one downmix channel; and
generating the output multi-channel representation of the spatial audio signal using the intermediate representation of the spatial audio signal by performing an upmixing operation, wherein the at least one downmix channel and the direction parameters are used in the upmixing operation.
23. A non-transitory storage medium having stored thereon a computer program for, when running on a computer, implementing a method for conversion of a multi-channel representation into a different output multi-channel representation of a spatial audio signal, the method comprising:
US11/742,502 2007-03-21 2007-04-30 Method and apparatus for conversion between multi-channel audio formats Active 2030-12-28 US8290167B2 (en)
US11/742,502 US8290167B2 (en) 2007-03-21 2007-04-30 Method and apparatus for conversion between multi-channel audio formats
CN 200880009025 CN101669167A (en) 2007-03-21 2008-02-01 Method and apparatus for conversion between multi-channel audio formats
EP20080707513 EP2130204A1 (en) 2007-03-21 2008-02-01 Method and apparatus for conversion between multi-channel audio formats
BRPI0808217 BRPI0808217A2 (en) 2007-03-21 2008-02-01 "method and equipment for conversion between multi-channel audio formats"
KR20097019537A KR101195980B1 (en) 2007-03-21 2008-02-01 Method and apparatus for conversion between multi-channel audio formats
RU2009134474/08A RU2449385C2 (en) 2007-03-21 2008-02-01 Method and apparatus for conversion between multichannel audio formats
JP2009553931A JP4993227B2 (en) 2007-03-21 2008-02-01 Method and apparatus for conversion between multi-channel audio formats
PCT/EP2008/000830 WO2008113428A1 (en) 2007-03-21 2008-02-01 Method and apparatus for conversion between multi-channel audio formats
TW097109731A TWI369909B (en) 2007-03-21 2008-03-19 Method and apparatus for conversion between multi-channel audio formats
US12/530,645 Continuation-In-Part US8908873B2 (en) 2007-03-21 2008-02-01 Method and apparatus for conversion between multi-channel audio formats
US11/742,502 Active 2030-12-28 US8290167B2 (en) 2007-03-21 2007-04-30 Method and apparatus for conversion between multi-channel audio formats
US20100169103A1 (en) * 2007-03-21 2010-07-01 Ville Pulkki Method and apparatus for enhancement of audio reconstruction
US20100166191A1 (en) * 2007-03-21 2010-07-01 Juergen Herre Method and Apparatus for Conversion Between Multi-Channel Audio Formats
US20110064258A1 (en) * 2008-04-21 2011-03-17 Snaps Networks, Inc Electrical System for a Speaker and its Control
US20110103591A1 (en) * 2008-07-01 2011-05-05 Nokia Corporation Apparatus and method for adjusting spatial cue information of a multichannel audio signal
US20120008789A1 (en) * 2010-07-07 2012-01-12 Korea Advanced Institute Of Science And Technology 3d sound reproducing method and apparatus
US9570083B2 (en) 2013-04-05 2017-02-14 Dolby International Ab Stereo audio encoder and decoder
US9820073B1 (en) 2017-05-10 2017-11-14 Tls Corp. Extracting a common signal from multiple audio signals
US9913061B1 (en) * 2016-08-29 2018-03-06 The Directv Group, Inc. Methods and systems for rendering binaural audio content
US10531215B2 (en) * 2010-12-28 2020-01-07 Samsung Electronics Co., Ltd. 3D sound reproducing method and apparatus
US8249283B2 (en) * 2006-01-19 2012-08-21 Nippon Hoso Kyokai Three-dimensional acoustic panning device
US9014377B2 (en) * 2006-05-17 2015-04-21 Creative Technology Ltd Multichannel surround format conversion and generalized upmix
US8180062B2 (en) * 2007-05-30 2012-05-15 Nokia Corporation Spatial sound zooming
EP2205007B1 (en) * 2008-12-30 2019-01-09 Dolby International AB Method and apparatus for three-dimensional acoustic field encoding and optimal reconstruction
WO2011044064A1 (en) 2009-10-05 2011-04-14 Harman International Industries, Incorporated System for spatial extraction of audio signals
EP2346028A1 (en) 2009-12-17 2011-07-20 Fraunhofer-Gesellschaft zur Förderung der Angewandten Forschung e.V. An apparatus and a method for converting a first parametric spatial audio signal into a second parametric spatial audio signal
CN103811010B (en) * 2010-02-24 2017-04-12 弗劳恩霍夫应用研究促进协会 Apparatus for generating an enhanced downmix signal and method for generating an enhanced downmix signal
ES2472456T3 (en) * 2010-03-26 2014-07-01 Thomson Licensing Method and device for decoding a representation of an acoustic audio field for audio reproduction
EP2375779A3 (en) 2010-03-31 2012-01-18 Fraunhofer-Gesellschaft zur Förderung der Angewandten Forschung e.V. Apparatus and method for measuring a plurality of loudspeakers and microphone array
WO2012025580A1 (en) 2010-08-27 2012-03-01 Sonicemotion Ag Method and device for enhanced sound field reproduction of spatially encoded audio input signals
JP5567997B2 (en) * 2010-12-07 2014-08-06 日本放送協会 Acoustic signal comparison device and program thereof
KR101871234B1 (en) 2012-01-02 2018-08-02 삼성전자주식회사 Apparatus and method for generating sound panorama
JP2015509212A (en) * 2012-01-19 2015-03-26 コーニンクレッカ フィリップス エヌ ヴェ Spatial audio rendering and encoding
CN103379424B (en) * 2012-04-24 2016-08-10 华为技术有限公司 A kind of sound mixing method and multipoint control server
EP2733964A1 (en) 2012-11-15 2014-05-21 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Segment-wise adjustment of spatial audio signal to different playback loudspeaker setup
SG11201504368VA (en) * 2012-12-04 2015-07-30 Samsung Electronics Co Ltd Audio providing apparatus and audio providing method
US9478224B2 (en) 2013-04-05 2016-10-25 Dolby International Ab Audio processing system
US9892737B2 (en) 2013-05-24 2018-02-13 Dolby International Ab Efficient coding of audio scenes comprising audio objects
KR102033304B1 (en) 2013-05-24 2019-10-17 돌비 인터네셔널 에이비 Efficient coding of audio scenes comprising audio objects
US9495968B2 (en) 2013-05-29 2016-11-15 Qualcomm Incorporated Identifying sources from which higher order ambisonic audio data is generated
EP2814027B1 (en) 2013-06-11 2016-08-10 Harman Becker Automotive Systems GmbH Directional audio coding conversion
EP2830332A3 (en) * 2013-07-22 2015-03-11 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Method, signal processing unit, and computer program for mapping a plurality of input channels of an input channel configuration to output channels of an output channel configuration
ES2641538T3 (en) * 2013-09-12 2017-11-10 Dolby International Ab Multichannel audio content encoding
CN105637901B (en) * 2013-10-07 2018-01-23 杜比实验室特许公司 Space audio processing system and method
US9852737B2 (en) * 2014-05-16 2017-12-26 Qualcomm Incorporated Coding vectors decomposed from higher-order ambisonics audio signals
CN105657633A (en) 2014-09-04 2016-06-08 杜比实验室特许公司 Method for generating metadata aiming at audio object
US9774974B2 (en) * 2014-09-24 2017-09-26 Electronics And Telecommunications Research Institute Audio metadata providing apparatus and method, and multichannel audio data playback apparatus and method to support dynamic format conversion
EP3297298A1 (en) 2016-09-19 2018-03-21 A-Volute Method for reproducing spatially distributed sounds
WO1992015180A1 (en) 1991-02-15 1992-09-03 Trifield Productions Ltd. Sound reproduction system
US5208860A (en) 1988-09-02 1993-05-04 Qsound Ltd. Sound imaging method and apparatus
JPH07222299A (en) 1994-01-31 1995-08-18 Matsushita Electric Ind Co Ltd Processing and editing device for movement of sound image
RU2092979C1 (en) 1988-09-02 1997-10-10 Кью Саунд Лтд. Method for generating and locating the seeming sound source in three-dimensional space and device for its implementation
US5812674A (en) 1995-08-25 1998-09-22 France Telecom Method to simulate the acoustical quality of a room and associated audio-digital processor
JPH10304498A (en) 1997-04-30 1998-11-13 Kawai Musical Instr Mfg Co Ltd Stereophonic extension device and sound field extension device
US5870484A (en) 1995-09-05 1999-02-09 Greenberger; Hal Loudspeaker array with signal dependent radiation pattern
US5873059A (en) 1995-10-26 1999-02-16 Sony Corporation Method and apparatus for decoding and changing the pitch of an encoded speech signal
RU2129336C1 (en) 1992-11-02 1999-04-20 Фраунхофер Гезелльшафт цур Фердерунг дер Ангевандтен Форшунг Е.Фау Method for transmission and/or storage of digital signals of more than one channel
US5909664A (en) 1991-01-08 1999-06-01 Ray Milton Dolby Method and apparatus for encoding and decoding audio information representing three-dimensional sound fields
EP1016320A2 (en) 1997-07-16 2000-07-05 Dolby Laboratories Licensing Corporation Method and apparatus for encoding and decoding multiple audio channels at low bit rates
WO2001082651A1 (en) 2000-04-19 2001-11-01 Sonic Solutions Multi-channel surround sound mastering and reproduction techniques that preserve spatial harmonics in three dimensions
WO2002007481A2 (en) 2000-07-19 2002-01-24 Koninklijke Philips Electronics N.V. Multi-channel stereo converter for deriving a stereo surround and/or audio centre signal
US6343131B1 (en) 1997-10-20 2002-01-29 Nokia Oyj Method and a system for processing a virtual acoustic environment
JP2003274492A (en) 2002-03-15 2003-09-26 Nippon Telegr & Teleph Corp <Ntt> Stereo acoustic signal processing method, stereo acoustic signal processor, and stereo acoustic signal processing program
US6628787B1 (en) 1998-03-31 2003-09-30 Lake Technology Ltd Wavelet conversion of 3-D audio signals
US6694033B1 (en) 1997-06-17 2004-02-17 British Telecommunications Public Limited Company Reproduction of spatialized audio
US6718039B1 (en) 1995-07-28 2004-04-06 Srs Labs, Inc. Acoustic correction apparatus
US20040091118A1 (en) 1996-07-19 2004-05-13 Harman International Industries, Incorporated 5-2-5 Matrix encoder and decoder system
US20040151325A1 (en) 2001-03-27 2004-08-05 Anthony Hooley Method and apparatus to create a sound field
WO2004077884A1 (en) 2003-02-26 2004-09-10 Helsinki University Of Technology A method for reproducing natural or modified spatial impression in multichannel listening
US6836243B2 (en) 2000-09-02 2004-12-28 Nokia Corporation System and method for processing a signal being emitted from a target signal source into a noisy environment
US20050053242A1 (en) 2001-07-10 2005-03-10 Fredrik Henn Efficient and scalable parametric stereo coding for low bitrate applications
WO2005101905A1 (en) 2004-04-16 2005-10-27 Coding Technologies Ab Scheme for generating a parametric representation for low-bit rate applications
US20050249367A1 (en) 2004-05-06 2005-11-10 Valve Corporation Encoding spatial data in a multi-channel sound file for an object in a virtual environment
WO2005117483A1 (en) 2004-05-25 2005-12-08 Huonlabs Pty Ltd Audio apparatus and method
US20060004583A1 (en) 2004-06-30 2006-01-05 Juergen Herre Multi-channel synthesizer and method for generating a multi-channel output signal
WO2006003813A1 (en) 2004-07-02 2006-01-12 Matsushita Electric Industrial Co., Ltd. Audio encoding and decoding apparatus
US20060093152A1 (en) 2004-10-28 2006-05-04 Thompson Jeffrey K Audio spatial environment up-mixer
US20060093128A1 (en) 2004-10-15 2006-05-04 Oxford William V Speakerphone
TW200629240A (en) 2004-12-23 2006-08-16 Motorola Inc Method and apparatus for audio signal enhancement
JP2006237839A (en) 2005-02-23 2006-09-07 Oki Electric Ind Co Ltd Voice conference device
US7110953B1 (en) 2000-06-02 2006-09-19 Agere Systems Inc. Perceptual coding of audio signals using separated irrelevancy reduction and redundancy reduction
EP1761110A1 (en) 2005-09-02 2007-03-07 Ecole Polytechnique Fédérale de Lausanne Method to generate multi-channel audio signals from stereo signals
KR20070042145A (en) 2004-07-14 2007-04-20 코닌클리케 필립스 일렉트로닉스 엔.브이. Audio channel conversion
US7243073B2 (en) 2002-08-23 2007-07-10 Via Technologies, Inc. Method for realizing virtual multi-channel output by spectrum analysis
US20070269063A1 (en) * 2006-05-17 2007-11-22 Creative Technology Ltd Spatial audio coding based on universal spatial cues
US7668722B2 (en) * 2004-11-02 2010-02-23 Coding Technologies Ab Multi parametrisation based multi-channel reconstruction
JP4295798B2 (en) * 2005-06-21 2009-07-15 独立行政法人科学技術振興機構 Mixing apparatus, method, and program
2008-02-01 BR BRPI0808217 patent/BRPI0808217A2/en active Search and Examination
JPH06506092A (en) 1991-02-15 1994-07-07
EP1016320B1 (en) 1997-07-16 2002-03-27 Dolby Laboratories Licensing Corporation Method and apparatus for encoding and decoding multiple audio channels at low bit rates
RU2234819C2 (en) 1997-10-20 2004-08-20 Нокиа Ойй Method and system for transferring characteristics of ambient virtual acoustic space
EP1275272A1 (en) 2000-04-19 2003-01-15 Sonic Solutions Multi-channel surround sound mastering and reproduction techniques that preserve spatial harmonics in three dimensions
JP2004504787A (en) 2000-07-19 2004-02-12 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィKoninklijke Philips Electronics N.V. Multi-channel stereo converter to obtain stereo surround and / or audio center signal
JP2006087130A (en) 2001-07-10 2006-03-30 Coding Technologies Ab Efficient and scalable parametric stereo encoding for low bit rate audio encoding
JP2007533221A (en) 2004-04-16 2007-11-15 コーディング テクノロジーズ アクチボラゲット Generation of parametric representations for low bit rates
US20070127733A1 (en) 2004-04-16 2007-06-07 Fredrik Henn Scheme for Generating a Parametric Representation for Low-Bit Rate Applications
KR20070001227A (en) 2004-04-16 2007-01-03 코딩 테크놀러지스 에이비 Scheme for generating a parametric representation for low-bit rate applications
US7853022B2 (en) * 2004-10-28 2010-12-14 Thompson Jeffrey K Audio spatial environment engine
Allen, Jont B., "Image Method for Efficiently Simulating Small-Room Acoustics," 1979, Journal of the Acoustical Society of America, vol. 65, pp. 943-950.
Atal, B.S., et al., "Perception of Coloration in Filtered Gaussian Noise-Short-Time Spectral Analysis by the Ear," Aug. 21-28, 1962, Fourth International Congress on Acoustics, Copenhagen.
Atal, B.S., et al., "Perception of Coloration in Filtered Gaussian Noise—Short-Time Spectral Analysis by the Ear," Aug. 21-28, 1962, Fourth International Congress on Acoustics, Copenhagen.
Avendano, Carlos, "A Frequency-Domain Approach to Multichannel Upmix," Jul./Aug. 2004, Journal of the Audio Engineering Society, vol. 52, No. 7/8.
Avendano, Carlos, et al., "Ambience Extraction and Synthesis from Stereo Signals for Multi-Channel Audio Up-Mix," 2002, Creative Advanced Technology Center.
Bech, Soren, "Timbral Aspects of Reproduced Sound in Small Rooms. I," Mar. 1995, Journal of the Acoustical Society of America, vol. 97, No. 3, pp. 1717-1726.
Bilsen, Frans A., "Pitch of Noise Signals: Evidence for a 'Central Spectrum'," 1977, Journal of the Acoustical Society of America, vol. 61, No. 1.
Bilsen, Frans A., "Pitch of Noise Signals: Evidence for a 'Central Spectrum'," 1977, Journal of the Acoustical Society of America, vol. 61, No. 1.
Bitzer, Joerg, et al., "Superdirective Microphone Arrays," in M. Brandstein, D. Ward edition: Microphone Arrays-Signal Processing Techniques and Applications, Chapter 2, Springer Berlin 2001, ISBN: 978-3-540-41953-2.
Bitzer, Joerg, et al., "Superdirective Microphone Arrays," in M. Brandstein, D. Ward edition: Microphone Arrays—Signal Processing Techniques and Applications, Chapter 2, Springer Berlin 2001, ISBN: 978-3-540-41953-2.
Bronkhorst, A.W., et al., "The Effectof Head-Induced Interaural Time and Level Differences on Speech Intelligibility in Noise," 1988, Journal of the Acoustical Society of America, vol. 83, pp. 1508-1516.
Bruggen, Marc, et al., "Coloration and Binaural Decoloration in Natural Environments," Apr. 19, 2001, Acustica, vol. 87, pp. 400-406.
Chen, Jingdong, et al., "Time Delay Estimation in Room Acoustic Environments: An Overview," 2006, EURASIP Journal on Applied Signal Processing, vol. 2006, Articale 26503, pp. 1-19.
Culling, John F., et al., "Dichotic Pitches as Illusions of Binaural Unmasking," Jun. 1998, Journal of the Acoustical Society of America, pp. 3509-3526.
Daniel, J. et al.; "Ambisonics Encoding of Other Audio Formats for Multiple Listening Conditions"; Sep. 26-29, 1998; Presented at the105th AES Convention, San Franciso, California, 29 pages.
Dressler, Roger, "Dolby Surround Pro Logic II Decoder-Principles of Operation," Aug. 2004, Dolby Publication, http:.www.dolby.com/assets/pdf/tech-library/209-Dolby-Surround-Pro-Logic-II-Decoder-Principles-of-Operation.pdf.
Dressler, Roger, "Dolby Surround Pro Logic II Decoder—Principles of Operation," Aug. 2004, Dolby Publication, http:.www.dolby.com/assets/pdf/tech—library/209—Dolby—Surround—Pro—Logic—II—Decoder—Principles—of—Operation.pdf.
Elko, Gary W., "Superdirectional Microphone Arrays," in S.G. Gay, J. Benesty edition: "Acoustic signal Processing for Telecommunication," Chapter 10, Kluwer Academic Press, ISBN: 978-0792378143.
European Patent Office Correspondence, mailed Feb. 24, 2011, in related European Patent Application No. 08707513.1-2225, 6 pages.
Faller, Christof, "Multiple-Loudspeaker Playback of Stereo Signals," Nov. 2006, Journal of the Audio Engineering Society, vol. 54, No. 11.
Faller, Christof, et al., "Source Localization in Complex Listening Situations: Selection of Binaural Cues Based on Interaural Coherence," Nov. 2004, Journal of the Acoustical Society of America, vol. 116, No. 5, pp. 3075-3089.
Gerzon, Michael A., "Periphony: With-Height Sound Reproduction," Jan./Feb. 1973, Journal of the Audio Engineering Society, vol. 21, No. 1, pp. 2-10.
Griesinger, David, "Multichannel Matrix Surround Decoders for Two-Eared Listeners," Nov. 8-11, 1996; Journal of the Audio Engineering Society, 101st AES Convention, Los Angeles, California, Preprint 4402.
Herre, et al.; "The Reference Model Architecture for MPEG Spatial Audio Coding": May 28, 2005, AES Convention paper, pp. 1-13; New York, NY, XP009059973.
Itu-R. Rec. BS.775-1, "Multi-Channel Stereophonic Sound System With or Without Accompanying Picture," 1992-1994, International Telecommunications Union, Geneva, Switzerland.
Laborie, Arnaud, et al., "Designing High Spatial Resolution Microphones," Oct. 28-31, 2004, Journal of the Audio Engineering Society, Convention Paper 6231, San Francisco, CA.
Lipshitz, Stanley P., "Stereo Microphone Techniques . . . Are the Purists Wrong?," Sep. 1986, Journal of the Audio Engineering Society, vol. 34, No. 9, pp. 716-744.
Merimaa, Juha, et al., "Spatial Impulse Response Rendering I: Analysis and Synthesis," Dec. 2005, Journal of the Audio Engineering Society, vol. 53, No. 12, pp. 1115-1127.
Nelisse, H., et al., "Characterization of a Diffuse Field in a Reverberant Room," Jun. 1997, Journal of the Acoustical Society of America, vol. 101, No. 6, pp. 3417-3524.
Okano, Toshiyuki, et al., "Relations Among Interaural Cross-Correlation Coefficient (IACCe), Lateral Fraction (LFe), and Apparent Source Width (ASW) in Concert Halls," Jul. 1998, Journal of the Acoustical Society of America, pp. 255-265.
Pulkki, V. , "Applications of Directional Audio Coding in Audio", 19th International Congress of Acoustics, International Commission for Acoustics, retrieved online from http://decoy.iki.fi/dsound/ambisonic/motherlode/source/rba-15-2002.pdf, Sep. 2007, 6 pages.
Pulkki, V., "Directioinal Audio Coding in Spatial Sound Reproduction and Stereo Upmixing;" Jun. 30-Jul. 2, 2006, Proceedings of the AES 28th International Conference, pp. 251-258, Pitea, Sweden.
Pulkki, Ville, "Virtual Sound Source Positioning Using Vector Base Amplitude Panning," Jun. 1997, Journal of the Audio Engineering Society, vol. 45, No. 6.
Pulkki, Ville, et al., "Directional Audio Coding: Filterbank and STFT-based Design," May 20-23, 2006, Journal of the Audio Engineering Society, AES 120th Convention, Paris, France, Preprint 6658.
Pulkki, Ville, et al., "Spatial Impulse Response Rendering II: Reproduction of Diffuse Sound and Listening Tests," Jan./Feb. 2006, Journal of the Audio Engineering Society, vol. 54, No. 1/2, pp. 3-20.
Schulein, Robert B., "Microphone Considerations in Feedback-Prone Environments," Jul./Aug. 1976, Journal of the Audio Engineering Society, vol. 24, No. 6.
Simmer, K. Uwe, et al., "Post Filtering Techniques," in M. Brandstein, D. Ward edition: Microphone Arrays-Signal Processing Techniques and Applications, Chapter 3, Springer Berlin 2001, ISBN: 978-3-540-41953-2.
Simmer, K. Uwe, et al., "Post Filtering Techniques," in M. Brandstein, D. Ward edition: Microphone Arrays—Signal Processing Techniques and Applications, Chapter 3, Springer Berlin 2001, ISBN: 978-3-540-41953-2.
Streicher, Ron, et al., "Basic Stereo Microphone Perspectives-A Review," Jul./Aug. 1985, Journal of the Audio Engineering Society, vol. 33, No. 7/8.
Streicher, Ron, et al., "Basic Stereo Microphone Perspectives—A Review," Jul./Aug. 1985, Journal of the Audio Engineering Society, vol. 33, No. 7/8.
The Russian Decision to grant mailed Sep. 7, 2010 in related Russian Patent Application No. 2009134471/09(048571); 10 pages.
Villemoes, Lars, et al., "MPEG Surround: The Forthcoming ISO Standard for Spatial Audio Coding," Jun. 30-Jul. 2, 2006, AES 28th International Conference, Pitea, Sweden.
Zielinski, Slawomir K., "Comparison of Basic Audio Quality and Timbral and Spatial Fidelity Changes Caused by Limitation of Bandwidth and by Down-mix Algorithms in 5.1 Surround Audio Systems," Mar. 2005, Journal of the Audio Engineering Society, vol. 53, No. 3.
US8908873B2 (en) 2007-03-21 2014-12-09 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Method and apparatus for conversion between multi-channel audio formats
US9015051B2 (en) * 2007-03-21 2015-04-21 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Reconstruction of audio channels with direction parameters indicating direction of origin
US9872091B2 (en) 2008-04-21 2018-01-16 Caavo Inc Electrical system for a speaker and its control
US8588431B2 (en) * 2008-04-21 2013-11-19 Snap Networks, Inc. Electrical system for a speaker and its control
US9025775B2 (en) * 2008-07-01 2015-05-05 Nokia Corporation Apparatus and method for adjusting spatial cue information of a multichannel audio signal
US10163449B2 (en) 2013-04-05 2018-12-25 Dolby International Ab Stereo audio encoder and decoder
US10129680B2 (en) 2016-08-29 2018-11-13 The Directv Group, Inc. Methods and systems for rendering binaural audio content
RU2604342C2 (en) 2016-12-10 Device and method of generating output audio signals using object-oriented metadata
RU2329548C2 (en) 2008-07-20 Device and method of multi-channel output signal generation or generation of diminishing signal
US7447317B2 (en) 2008-11-04 Compatible multi-channel coding/decoding by weighting the downmix channel
JP5185340B2 (en) 2013-04-17 Apparatus and method for displaying a multi-channel audio signal
US8625808B2 (en) 2014-01-07 Methods and apparatuses for encoding and decoding object-based audio signals
JP4606507B2 (en) 2011-01-05 Spatial downmix generation from parametric representations of multichannel signals
TWI424756B (en) 2014-01-21 Binaural rendering of a multi-channel audio signal
JP6615936B2 (en) 2019-12-04 Method and apparatus for decoding audio field representation for audio playback
US7489788B2 (en) 2009-02-10 Recording a three dimensional auditory scene and reproducing it for the individual listener
RU2409911C2 (en) 2011-01-20 Decoding binaural audio signals
TWI305639B (en) 2009-01-21 Apparatus and method for generating a multi-channel output signal
ES2339888T3 (en) 2010-05-26 Audio coding and decoding.
US9397771B2 (en) 2016-07-19 Method and apparatus for encoding and decoding successive frames of an ambisonics representation of a 2- or 3-dimensional sound field
EP1817767B1 (en) 2015-11-11 Parametric coding of spatial audio with object-based side information
JP5156386B2 (en) 2013-03-06 Compact side information for parametric coding of spatial speech
EP2198632B1 (en) 2014-03-19 Method and apparatus for generating a binaural audio signal
AU2008215231B2 (en) 2010-02-18 Methods and apparatuses for encoding and decoding object-based audio signals
CA2077668C (en) 2001-02-27 Decoder for variable-number of channel presentation of multidimensional sound fields
JP4625084B2 (en) 2011-02-02 Shaped diffuse sound for binaural cue coding method etc.
JP5624967B2 (en) 2014-11-12 Apparatus and method for generating a multi-channel synthesizer control signal and apparatus and method for multi-channel synthesis
JP5455657B2 (en) 2014-03-26 Method and apparatus for enhancing speech reproduction
EP1971978B1 (en) 2010-08-04 Controlling the decoding of binaural audio signals
US7853022B2 (en) 2010-12-14 Audio spatial environment engine
US8379868B2 (en) 2013-02-19 Spatial audio coding based on universal spatial cues
CN101065797B (en) 2011-07-27 Dynamic down-mixer system
Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWAN
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PULKKI, VILLE;HERRE, JUERGEN;REEL/FRAME:019606/0219;SIGNING DATES FROM 20070522 TO 20070611
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PULKKI, VILLE;HERRE, JUERGEN;SIGNING DATES FROM 20070522 TO 20070611;REEL/FRAME:019606/0219