FIELD

An embodiment of the invention pertains generally to the encoding and decoding of an audio signal, and the use of metadata, associated with the encoded signal, during playback of the decoded signal, to improve quality of playback in various types of consumer electronics end user devices. Other embodiments are also described.

BACKGROUND

Digital audio content appears in many instances, including for example music and movie files. In most instances, an audio signal is encoded, where the encoding need not necessarily be for purposes of data-rate reduction but could simply be for purposes of format conversion, to enable the storage or transmission of a resulting media file or stream, thereby allowing numerous deliveries or transmissions to occur simultaneously (if needed). The media file or stream can be received in different types of end user devices, where the encoded audio signal is decoded before being presented to the consumer through either built-in or detachable speakers. This has helped fuel consumers' appetite for obtaining digital media over the Internet. Creators and distributors of digital audio content (programs) have several industry standards at their disposal, which can be used for encoding and decoding audio content. These include Digital Audio Compression Standard (AC-3, E-AC-3), Revision B, Document A/52B, 14 Jun. 2005 published by the Advanced Television Systems Committee, Inc. (the “ATSC Standard”), European Telecommunication Standards Institute, ETSI TS 101 154 Digital Video Broadcasting (DVB) based on MPEG-2 Transport Stream in ISO/IEC 13818-7, Advanced Audio Coding (AAC) (“MPEG-2 AAC Standard”), and ISO/IEC 14496-3 (“MPEG-4 Audio”), published by the International Standards Organization (ISO).

Audio content may be decoded and then processed (rendered) differently than it was originally mastered. For example, a mastering engineer could record an orchestra or a concert such that upon playback it would sound (to a listener) as if the listener were sitting in the audience of the concert, i.e. in front of the band or orchestra, with the applause being heard from behind. The mastering engineer could alternatively make a different rendering (of the same concert), so that, for example upon playback the listener would hear the concert as if he were on stage (where he would hear the instruments “around him”, and the applause “in front”). This is also referred to as creating a different perspective for the listener in the playback room, or rendering the audio content for a different “listening location” or different playback room.

Audio content may also be rendered for different acoustic environments, e.g. playback through a headset, a smartphone speakerphone, or the built-in speakers of a tablet computer, a laptop computer, or a desktop computer. In particular, object based audio playback techniques are now available where an individual digital audio object, which is a digital audio recording of, e.g. a single person talking, an explosion, applause, or background sounds, can be played back differently over any one or more speaker channels in a given acoustic environment.

However, tonal balance as heard by a listener is affected when a previously mixed recording (of certain audio content) is then rendered into a different acoustic environment, or rendered from a different listener perspective. To alleviate such tonal imbalance, mastering engineers apply equalization (EQ) or spectral shaping to a digital audio signal, in order to optimize the audio signal for a particular acoustic environment or for a particular listener perspective. For example, rendering a motion picture file for playback in a large movie theater may call for certain EQ to be applied (to the digital audio tracks of the motion picture file) to prevent the resulting sound from being too bright during playback. But rendering the file for playback through a home theater system, e.g. as a DVD file or an Internet streaming movie file, calls for a different EQ because of the smaller room size (and other factors).

SUMMARY

In mixing and mastering or production of audio, it is common to apply two basic tools, dynamic range control or compression (DRC) and equalization (EQ), to achieve a desired level (e.g., loudness) and tonal balance. However, the consumer centered playback scenario is an unknown during production. An embodiment of the invention is a production or distribution system (e.g., a server system) that produces EQ values which are part of metadata of an encoded, digital audio content (or audio recording) file. The EQ values may define linear filtering (spectral shaping) that is to be applied to the decoded original audio recording, during playback. This application of EQ may be defined to be independent of any downmix. The spectral shaping is applied after an audio recording has been extracted by a decoder from the encoded content file; this may take place prior to a specified downmix being formed, after the downmix has been performed, or both prior to and after the downmix. The linear filtering defined by the metadata-based EQ values is designed to adjust the balance between frequency components within the audio recording, strengthening or weakening the energy of specific frequency bands, in order to better suit a particular playback acoustic environment or listening perspective. Accordingly, this enables the playback of spectrally adjusted or linear filtered audio content, where the filtering was specified at the encoding stage and intended to improve playback sound in a particular acoustical environment (e.g., a living room which is smaller than public movie theater) or for a particular sound rendering system (e.g., having a particular audio amplifier gain and speaker sensitivity). The audio content file may be for example a moving picture file, e.g. an MPEG movie file, an audio-only file, e.g. an AAC file, or a file having any suitable multimedia format. For instance, the metadata associated with the encoded digital audio recording may be carried in a number of extension fields of MPEG-D DRC. A metadata-based approach enables a unified sound program content distribution format because only a single, base version of the audio recording is needed, with the ability to deliver multiple rendered versions of the base version for different playback scenarios (as supported by the EQ sequences in the metadata.)

Various formats are possible for storing or carrying the EQ values within the metadata of an audio content file. For example, the metadata may be defined to contain one or more sequences of EQ values. See the table below for example portions of an EQ sequence. Each element of a sequence may contain a value that defines some aspect of a spectral shaping (filtering) operation that is to be performed upon a particular decoded channel or object. The EQ values have been selected for a particular rendering of the encoded digital audio content. A sequence of EQ values may define equalization filtering that is to be applied to the entirety of a piece of sound program content (in the decoded audio recording). Each EQ sequence in the metadata of a given file (referring to the same audio content in the file), may include the instructions for filtering a respective group of one or more channels or objects. An EQ set (of one or more EQ sequences) may define the EQ filtering that is applied during playback of a song through a headset, while another EQ set may define the EQ filtering that is to be applied for playback of the same song through a speaker dock. The filtering defined in a given EQ sequence is thus associated with an EQ group of one or more of the original audio channels or audio objects, and may be independent of any downmix that is to also be applied to the channels or objects of that EQ group.

The EQ values (or EQ instructions or payload) may define filtering that is to be applied in the frequency (or sub-band) domain. For example, an EQ payload may have gain values for two or more predefined frequency bands, similar to a graphic equalizer setting, e.g. treble, mid, and bass. In that case, the EQ filtering operation may multiply a frequency domain representation of the decoded audio content by the gain values. The EQ payload may alternatively, or in addition, define filtering that is to be applied in the time domain. For example, the EQ payload may contain the tap coefficients of a digital filter. In that case, an adaptive or variable digital filter may be synthesized (in the decoder stage) through which the decoded audio content is passing during playback, through the filter, in accordance with the EQ payload. In that case, the EQ filtering operation may be to pass a time domain representation of the audio content through the synthesized digital filter, as the latter is being updated during playback.

As suggested above, the application of EQ filtering may change during playback, for example as often as every frame of the audio content. But the EQ may also be designed to change at a higher layer. For example, when an audio object comes into a scene, the EQ that is applied to select audio channels may need to change, so as to better accommodate the sound of the audio object. The sequence format may therefore allow the metadata-embedded EQ values to change over time, in synch with the encoded audio content. Techniques are also described here (for the EQ metadata) that may enable for example a new EQ to fade in and an old EQ to fade out, or abrupt switching on or off of EQ (during playback).

The above summary does not include an exhaustive list of all aspects of the present invention. It is contemplated that the invention includes all systems and methods that can be practiced from all suitable combinations of the various aspects summarized above, as well as those disclosed in the Detailed Description below and particularly pointed out in the claims filed with the application. Such combinations have particular advantages not specifically recited in the above summary.

