FIELD

This disclosure relates to the field of systems for communicating multiple streams of audio signals; and more specifically, to processing systems designed to encode and mix multiple streams of audio signals for transmission over a channel limited link, and processing systems designed to decode and separate a received mixed audio signal into multiple streams to enable independent control of the streams. Other aspects are also described.

BACKGROUND

When playing music, carrying on a telephone call, or listening to other audio content using a smartphone or other devices, another audio stream may “barge-in.” For example, a playback of stereo music may be interrupted by a response from a virtual assistant, or by other types of audio notifications or alerts received from a server or generated by the smartphone. It is desirable for the smartphone to provide a more pleasing listening experience to a user when there are multiple audio streams.

SUMMARY

A user may listen to audio streams through an earphone that receives the audio streams via a wireless or wired link from an audio source device, such as a smartphone. The communication link between the smartphone and the earphone may be bandwidth or channel limited, such as in a BLUETOOTH link. As a result, the smartphone may mix audio streams with different bandwidth requirements, such as the stereo music encoded on two channels and the virtual assistant response encoded on one channel, into a mixed stream with a signal bandwidth that allows the mixed stream to be transmitted over the channel limited link to the earphone. In other situations, multiple earphones may receive the mixed stream from a single smartphone. It may be desirable to selectively enable the mixed stream on the earphones. To provide the desired intelligibility, audio quality and privacy, and to improve the overall listening experiences to consumers of audio signals communicated over a channel limited link, a flexible approach to encode and mix multiple audio signals into a mixed stream, and to decode and separate a received mixed stream into its constituent audio signals is performed.

When a user listens to a mixed stream of audio signals on a playback device communicated from a host device, such as an earphone linked to a smartphone, it is desirable for some characteristics of the constituent audio signals of the mixed stream, such as their gain, processing latency, or masking capability to be independently controlled. In one scenario, independent gain control of multiple audio signals in a mixed stream improves the intelligibility of one audio signal relative to another audio signal when playing the mixed stream. For example, when the playback of stereo music is interrupted by a virtual assistant response, the volume of the stereo music may fade to accommodate the audio of the virtual assistant response, in a process referred to as “barge-in” ducking. In another scenario, independent latency control of multiple audio signals allows an audio signal to bypass signal processing performed on another audio signal of the mixed stream. For example, the virtual assistant response may bypass noise suppression, frequency equalization, or other audio processing performed on stereo music to reduce the processing latency for the virtual assistant response with no effect on its audio quality. In another scenario, independent masking capability allows an audio signal of a mixed stream to be selectively masked to protect the privacy of a user. For example, when the host device transmits a mixed stream of music and virtual assistant response to multiple earphones, the virtual assistant response may be masked to all earphones except for the earphone from which a user solicited the virtual assistant response, in what is referred to as a splitter mode.

In one embodiment, to provide independent control of constituent audio signals of a mixed stream, the host device may encode the constituent audio signals to enable a complete separation of the constituent audio signals when the mixed stream is decoded on the playback device. The gains of the constituent audio signals may be independently controlled before they are mixed to increase the intelligibility of one audio signal relative to another audio signal at the playback device. The ability to separate the constituent audio signals from the mixed signals at the playback device allows the processing operations performed on the constituent audio signals and the path latencies associated with the processing operations to be independently chosen. In addition, in applications where the mixed stream is transmitted from a single host device to multiple playback devices, the constituent audio signals may be selectively masked on a playback device to increase user privacy.

A system and method for decoding and separating constituent audio signals of a mixed stream to enable independent control of gain, latency, or masking capability of the constituent audio signals is disclosed. A device such as a playback audio device receives audio frames from a host device over a communication link. The audio frames contain a mixed audio signal of a converted playback audio signal and a notification audio signal. The converted playback audio signal and the notification audio signal may have independent gains. The device separates the mixed audio signal into its constituent converted playback audio signal and notification audio signal. The device then remixes the converted playback audio signal and the notification audio signal to generate a remixed signal. The device determines whether the notification audio signal is to be selectively masked or played by the device among multiple devices that receive the same audio frames in parallel. If the notification audio signal is to be selectively played, the device plays the remixed audio signal. If the notification audio signal is to be selectively masked, the device plays the converted playback audio signal.

The above summary does not include an exhaustive list of all aspects of the present invention. It is contemplated that the invention includes all systems and methods that can be practiced from all suitable combinations of the various aspects summarized above, as well as those disclosed in the Detailed Description below and particularly pointed out in the claims filed with the application. Such combinations have particular advantages not specifically recited in the above summary.

