TECHNOLOGY

The present Application relates to audio signal processing. More specifically, embodiments of the present invention relate to rendering audio objects in spatially encoded audio signals.

While some embodiments will be described herein with particular reference to that application, it will be appreciated that the invention is not limited to such a field of use, and is applicable in broader contexts.

BACKGROUND

Any discussion of the background art throughout the specification should in no way be considered as an admission that such art is widely known or forms part of common general knowledge in the field.

The new Dolby Atmos™ cinema system introduced the concept of a hybrid audio authoring, a distribution and playback representation that includes both audio beds (audio channels, also referred to static objects) and dynamic audio objects. In the present description, the term ‘audio objects’ relates to particular components of a captured audio input that are spatially, spectrally or otherwise distinct. Audio objects often originate from different physical sources. Examples of audio objects include audio such as voices, instruments, music, ambience, background noise and other sound effects such as approaching cars.

In the Atmos™ system, audio beds (or static objects) refer to audio channels that are meant to be reproduced at predefined, fixed loudspeaker locations. Dynamic audio objects, on the other hand, refer to individual audio elements that may exist for a defined duration in time and have spatial information describing certain properties of the object, such as its intended position, the object size, information indicating a specific subset of loudspeakers to be enabled for reproduction of the dynamic objects, and alike. This additional information is referred to as object metadata and allows the authoring of audio content independent of the end-point loudspeaker setup, since dynamic objects are not linked to specific loudspeakers. Furthermore, object properties may change over time, and consequently metadata can be time varying.

Reproduction of hybrid audio requires a renderer to transform the object-based audio representation to loudspeaker signals. A renderer takes as inputs (1) the object audio signals, (2) the object metadata, (3) the end-point loudspeaker setup, indicating the locations of the loudspeakers, and outputs loudspeaker signals. The aim of the renderer is to produce loudspeaker signals that result in a perceived object location that is equal to the intended location as specified by the object metadata. In the case that no loudspeaker is available at the intended position, a so-called phantom image is created by panning the object across two or more loudspeakers in the vicinity of the intended object position. In mathematical form, a conventional renderer can be described by a set of time-varying panning gains gi,j(t) being applied to a set of object audio signals xj(t) to result in a set of loudspeaker signals si(t):
si(t)=Σjgi,j(t)xj(t)  (Eq 1)

In this formulation, index i refers to a loudspeaker, and index j is the object index. The panning gains gi,j(t) result from the loudspeaker positions Piin the loudspeaker set P and time-varying object position metadata Mj(t)

Mj⁡(t)=[Xj⁡(t)Yj⁡(t)Zj⁡(t)](Eq⁢⁢2)

based on a panning law or panning function:
gi,j(t)=(P,Mj(t))  (Eq 3)

A wide range of methods of specifyingto compute panning gains for a given loudspeaker with index i and position P1have been proposed in the past. These include, but are not limited to, the sine-cosine panning law, the tangent panning law, and the sine panning law (cf. Breebaart, 2013 for an overview). Furthermore, multi-channel panning laws such as vector-based amplitude panning (VBAP) have been proposed for 3-dimensional panning (Pulkki, 2002).

Amplitude panning has shown to work well when applied to pair-wise panning across loudspeakers in the horizontal (left-right) plane that are symmetrically placed in terms of their azimuth. The maximum azimuth aperture angle between loudspeakers for panning to work well amounts to approximately 60 degrees, allowing a phantom image to be created between −30 and +30 degrees azimuth. Panning across loudspeakers lateral to the listener (front to rear in the listening frame), however, causes a variety of problems:When the listener is not exactly positioned in a desired audio ‘sweet spot’, or whenever loudspeakers are not exactly delay aligned at the listener's position, combing artifacts will arise when an object is panned across two loudspeakers. This combing effect deteriorates the perceived timbre of the phantom source, and results in a collapse of the spaciousness of the overall scene. Moreover, small changes in the orientation and position of the head will cause comb-filter notches and peaks to shift in frequency. As a result, the sweet spot in a multi-channel loudspeaker setup is often small and the perceived timbre strongly depends on the head orientation and position. This is sometimes referred to as ‘the rocking chair’ problem.In pair-wise panning using symmetrically-placed loudspeakers in front of the listener, the contribution of the two loudspeakers results in sound-source localization cues at the level of the listener's eardrums that closely correspond to those arising from the intended sound source location. This process does not work reliably for panning across loudspeakers in the front-to-rear direction. As a result, the perceived phantom source location can be ambiguous, or may be very different from the intended source location.Downmixing of rendered audio content (for example from Dolby Digital 5.1-ATSC A/52 standard-to stereo) causes an increase in the audio level of audio objects that are panned across front and surround loudspeakers. This is caused by the fact that panning laws are typically energy preserving, i.e.:
1=Σigi,j2(Eq 4)

When the corresponding loudspeaker signals are downmixed electrically, a gain buildup will occur because for any gains 0≤gi,j≤1:
Σigi,j≥√{square root over (Σigi,j2)}  (Eq 5)

The limitations of existing audio systems are particularly relevant for Dolby Digital 5.1 playback, and/or for loudspeaker configurations with 4 overhead loudspeakers such as 5.1.4 or 7.1.4. For such loudspeaker configurations, (dynamic) objects with metadata indicating a position in the middle of the room, or in the middle of the ceiling plane will typically be phantom-imaged between pair-wise remotely placed front and rear loudspeakers. Furthermore, side-surround channels may be produced as phantom images as well. An example of such phantom-imaging problem is visualized inFIG. 1, which illustrates a square room with four corner loudspeakers labeled ‘Lf’, ‘Rf’, ‘Ls’, and ‘Rs’, which are placed in the corners of the square room. A fifth center loudspeaker labeled ‘C’ is positioned directly in front of a listener's position (which corresponds roughly to the center of the room). An audio object with metadata coordinates (x=0, y=0.4) as depicted by the circle labeled ‘object’ is typically amplitude panned between loudspeakers labeled ‘Lf’ and ‘Ls’, as indicated by the arrows originating from ‘object’. Furthermore, if the content comprises more than five channels, for example also comprising a right side-surround channel (dashed-line loudspeaker icon labeled ‘Rss’ inFIG. 1), the signal associated with that channel may be reproduced by loudspeakers labeled ‘Rf’ and ‘Rs’ to preserve the spatial intent of that particular channel.

Amplitude panning as depicted inFIG. 1can be thought of as compromising timbre and sweet spot size against maintaining spatial artistic intent for sweet-spot listening.

Note that with a 7-channel loudspeaker setup (e.g. including ‘Lss’ and ‘Rss’ loudspeakers), the content depicted inFIG. 1would have significantly less phantom-imaging applied. In particular, the ‘Rss’ channel would be reproduced by a dedicated ‘Rss’ loudspeaker, while the object at y=0.4 would be reproduced mostly by the ‘Lss’ loudspeaker, with only a small amount of leakage to the ‘Lf’ loudspeaker.

There is a desire to mitigate the limitations imposed by prior-art amplitude panning.

SUMMARY OF THE INVENTION

In accordance with a first aspect of the present invention there is provided a method of rendering an audio signal for playback in an audio environment defined by a target loudspeaker system, the audio signal including audio data relating to an audio object and associated position data indicative of an object position, the method including the steps of:

a. receiving the audio signal;

b. receiving loudspeaker layout data for the target loudspeaker system;

c. receiving control data indicative of a position modification to be applied to the audio object in the audio environment;

d. in response to the position data, loudspeaker layout data and control data, generating rendering modification data; and

e. rendering the audio signal with the rendering modification data to output the audio signal with the audio object at a modified object position that is between loudspeakers within the audio environment.

In one embodiment each loudspeaker in the loudspeaker system is driven with a drive signal and the rendering modification data includes a modified drive signal for one or more of the loudspeakers. In one embodiment the drive signal is a function of position data and the modified drive signal is generated by modifying the position data. In one embodiment the drive signal is a function of loudspeaker layout data and the modified drive function is generated by modifying the loudspeaker layout data. In one embodiment the drive signal is a function of a panning law and the modified drive function is generated by modifying the panning law.

In one embodiment the modified object position is in a front-rear direction within the audio environment. In one embodiment the modified object position is a position nearer to one or more loudspeakers in the audio environment than the object position. In one embodiment the modified object position is a position nearer to a closest loudspeaker in the audio environment relative to the object position.

In one embodiment the rendering is performed such that an azimuth angle of the audio object between the object position and modified object position from the perspective of a listener is substantially unchanged.

In one embodiment the audio environment includes a coordinate system and the position data and loudspeaker layout data includes coordinates in the coordinate system.

In one embodiment the control data determines a type of rendering modification data to be generated. In one embodiment the control data determines a degree of position modification to be applied to the audio object during the rendering of the audio signal.

In one embodiment the degree of position modification is dependent upon the loudspeaker layout data. Preferably the degree of position modification is dependent upon a number of surround loudspeakers in the target loudspeaker system.

In one embodiment the audio signal includes the control data. In one embodiment the control data is generated during an authoring of the audio signal.

In one embodiment the loudspeaker layout data includes data indicative of two surround loudspeakers. In another embodiment the loudspeaker layout data includes data indicative of four surround loudspeakers.

In accordance with a second aspect of the present invention there is provided a computer system configured to perform a method according to the first aspect.

In accordance with a third aspect of the present invention there is provided a computer program configured to perform a method according to the first aspect.

In accordance with a fourth aspect of the present invention there is provided a non-transitive carrier medium carrying computer executable code that, when executed on a processor, causes the processor to perform a method according to the first aspect.

In accordance with a fifth aspect of the present invention there is provided an audio content creation system including:

an input for receiving audio data from one or more audio input devices, the audio data including data indicative of one or more audio objects;

an audio processing module to process the audio data and, in response, generate an audio signal and associated metadata including object position data indicative of a spatial position of the one or more audio objects within a first audio environment; and

a control module configured to generate rendering control data to control the performing of audio object position modification to be performed on the audio signal during rendering of that signal in a second audio environment.

In one embodiment the rendering control data includes an instruction to perform audio object position modification on a subset of the one or more audio objects. In one embodiment the rendering control data includes an instruction to perform audio object position modification on each of the one or more audio objects.

In one embodiment the object position modification is dependent upon a type of audio object.

In one embodiment the object position modification is dependent upon a position of the one or more objects in the second audio environment.

In one embodiment the rendering control data determines a type of object position modification to be performed.

In one embodiment the rendering control data determines a degree of object position modification to be applied to the one or more audio objects.

In one embodiment the rendering control data includes an instruction not to perform audio object position modification on any one of the audio objects.

In accordance with a sixth aspect of the present invention there is provided an audio rendering system including:

an input configured to receive:an audio signal including object audio data relating to one or more audio objects and associated object position data indicative of a spatial position of the one or more audio objects within a first audio environment;loudspeaker layout data for a target loudspeaker system defining a second audio environment; andrendering control data; and

a rendering module configured to render the audio signal based on the rendering control data and, in response, output the audio signal in a second audio environment with the one or more audio objects at respective modified object positions within the second audio environment.

In one embodiment the modified object positions are between original object positions and a position of at least one loudspeaker in the second audio environment.

In accordance with a seventh aspect of the present invention there is provided an audio processing system including the audio content system according to the fifth aspect and the audio rendering system according to the sixth aspect.

