Mobile devices have become ubiquitous in the everyday life of the general consumer. No longer are cellular phones, electronic personal data assistants, and Internet-connected hand-held devices reserved for the elite. As these devices become ingrained in consumers' daily routines, the use of the devices in situations in which safety, convenience and even appropriateness have become issues. For example, drivers routinely attempt to text, email and talk on their phone while driving. “Hands-free” operation has made use of these devices somewhat more acceptable in certain instances, but the user experience is less than ideal.
For example, the present state of the art does not provide a method for a user to effectively “barge in” to a device operating with a speech recognition system and text-to-speech system, while the text-to-speech system is producing simulated speech. Because a built-in microphone may be used to detect any user utterance, speech recognition technology often cannot distinguish between a user's voice attempting to control the mobile communication device, the voice of the text-to-speech system, the user's voice when the user is not providing direction to the device, and other voices and other background sounds that are audible in the acoustic environment.
This problem is common in systems such as interactive voice response (IVR) phone trees, which have difficulty deciphering spoken requests that are not directly responsive to the phone tree prompts (such as user requests to be transferred to an “operator”), due to the noise floor of the environment or connection problems, or due to the limitations of modern speech recognition technology. Similar issues are found in “constant” listening applications that are continually screening ambient audio for commands.
This situation may be exacerbated over speakerphone systems—such as are common in automobiles for hands-free cell phone use—because the mobile communication device microphone may be picking up both the voice of the user and the voice of recorded prompts from the IVR system. Speakerphone systems also exacerbate the problem because their microphones do not discriminate among sounds in the acoustic environment, which sounds can include voices other than that of the user, as well as other background sounds such as road noise. Hence, “barging in” or “getting the attention” of a device by speaking a specifically predefined voice command is ineffective by virtue of the limited accuracy of conventional speech recognition.
Additionally, some mobile voice-controlled applications deliver audio content themselves, such as simulated speech (text-to-speech), music or video. Because many devices place the speaker used for delivering the audio content in close proximity to the microphone used for detecting voice commands, it is even more difficult for the device to hear and distinguish the user's voice over audio content it is delivering.
For “barging in” or “getting the attention” of a device via user voice command, the device must be constantly listening to all acoustic signals in the environment and attempting to detect a specifically predefined “barge in” command. Today's mobile devices and the applications that run on such devices perform much of their speech recognition using cloud-based services, so constantly listening can be prohibitively expensive for the user from a data usage standpoint under current mobile data plans offered by cell carriers. There is also a latency associated with transmitting every utterance and sound from the mobile device to the cloud for speech recognition processing, which makes the mobile device inadequately responsive to such spoken barge-in commands. And “barge-in” commands are rendered impotent if the data connection between the mobile device and the network is lost. Also, constant listening takes a dramatic toll on battery life.
What is needed, therefore, is a system and techniques that allow a user to effectively and reliably interrupt or “get the attention” of an application using audible user signals, under a variety of acoustic conditions and in the presence of competing audio signals or noise, and without requiring access to a data network.