1. Field of the Invention
The present invention relates to a system predicting acoustic confusability of text phrases and audio data that is speech-recognition system independent. In particular, the system predicts speech recognizer confusion where utterances can be represented by any combination of a text form (text phrase/spelled form) and an audio file (audio data/phrase). For example:
Case 1: acoustic confusability predicted between two text phrases; and
Case 2: acoustic confusability predicted between two utterances, each of which may be represented either as a text phrase or an audio file.
2. Description of the Related Art
Using Acoustic Confusability Information in Voice Application Software:
Voice application software use speech recognizers to provide a voice user interface. For example, a voice application providing voice messaging uses a speech recognizer to process user voice commands such as “play message” and “save message.” In another example, a voice application providing voice messaging uses a speech recognizer to voice enroll names in an address book. In the context of speech recognizers, spoken phrases are deemed confusable if they sound alike. Typical voice application software use information on when the speech recognizer will confuse spoken phrases (i.e., also referred to as acoustic confusability by the speech recognizer). The capability of predicting acoustic confusability can be used by a voice application, for example, to alert a user to choose a different name when voice enrolling names in the address book, thereby reducing the risk of inefficient or inaccurate voice command processing by the voice application.
Spoken phrases that control a voice user interface of a voice application can have very similar pronunciations, but actually refer to different actions or persons. Two spoken phrases can confuse a speech recognizer due to similarities in pronunciations of the two spoken phrases. For example, the speech recognizer might confuse spoken phrases such as “Jill” and “Phil,” although only half of the letters in the text form of utterances (words/phrases) “Jill” and “Phil” are the same. In another example, the speech recognizer might confuse “reply” spoken by one speaker with “repeat” spoken by another speaker having a dialect that reduces final syllables.
Some typical speech recognizers recognize a spoken phrase by comparing the spoken phrase to a designed list of utterances (words/phrases) represented as audio files. For example, one type of typical speech recognizer recognizes a spoken phrase by finding the most similar phrase in its list of phrases with a given spoken phrase. Avoiding adding phrases on the list that may be confusable with each other reduces the possibility that the speech recognizer will erroneously recognize a given spoken phrase (i.e., one that appears on the list) as a different spoken phrase on the list, a mistake known as a substitution error. Phrases can be added to the list as follows: providing utterances represented as text forms, providing utterances represented as audio files (by speaking and recording the spoken phrase), and a combination of text form and audio file of utterances. Speech recognizers include algorithms to predict the acoustic similarity of two spoken phrases. The typical algorithms compare sets of acoustic measures to classify sounds in the spoken phrases. However, these algorithms do not predict acoustic similarity between text forms of utterances, and thereby cannot compare one utterance represented as an audio file to a text form of another utterance to predict similarity of the one utterance represented as the audio file to spoken examples of the text form of the other utterance.
Although typical speech recognizers compare sets of acoustic measures to classify sounds in spoken phrases, text forms of utterances can also be used to predict confusability of the text forms when spoken. However, for some languages, such as English, pronunciation similarities may not be obvious from the text forms (i.e., spelling) of utterances because there may not be a direct correspondence between the spelling and the pronunciation. In contrast, similarities in pronunciation can be more apparent from comparisons between phonetic transcriptions of utterances (existing speech recognizers use various equivalent phonetic representations) than from examination of the spellings of the utterances. In a language, phonetic transcriptions represent pronunciations by using different symbols to represent each phoneme, or sound unit (i.e., string of phonemes or phonetic symbols). However, phonetic transcriptions alone, or strings of phonemes, are not sufficient to predict confusability for the following reasons: first, because speech recognizers do not compare phonetic symbols, but instead, compare sets of acoustic measures to classify sounds as spoken phrases. Second, in many instances, different phonemes, such as the vowel in “pin” and the vowel in “pen,” may be acoustically similar (the sounds are similar), but represented by phonetic symbols that are different. For example, phonetic transcriptions of the words “pin” and “pen” differ by 33% (one of three phonemes), but can still be confusable by a speech recognizer because of the acoustic similarity of the vowels.
Typical acoustic confusability methods used by speech recognizers have a disadvantage because the typical acoustic confusability methods compare audio files with audio files. As a result, typical acoustic confusability methods are only useful in an application where only voice utterances (utterances represented as audio files) are used. For example, if names can be entered into a voice-controlled address book only by voice enrollment, then typical methods can be used to detect confusability among the address book entries. However, the typical acoustic confusability methods would not be useful in an application where voice and text are mixed. If names can be entered into an address book either by text or by voice, a method is needed to compare a text name to a voice name, so that names from a text enrolled address book can reliably (and not confusingly) be added to a voice enrolled address book. In the case of entering a name by voice, a method is needed to predict acoustic confusability between an utterance represented as an audio file and an utterance represented as a text form to compare the newly added voice name to the text enrolled names already in the address book. In the case of entering a name by text, a method is needed to predict acoustic confusability between a an utterance represented as a text form and an utterance represented as an audio file to compare the newly added text name to the voice enrolled names already in the address book.
Typical acoustic confusability methods that compare audio files with audio files have another disadvantage as follows: speech recognition can be either speaker independent or speaker dependent. When providing text to a speech recognizer, the techniques used to recognize speech would be speaker independent. When providing audio files to a speech recognizer, the techniques used to recognize speech can be either speaker independent or speaker dependent. However, speech recognition by typical speech recognizers is likely to be speech recognizer dependent (i.e., speaker dependent) when the speech recognizer is provided a combination of text phrases and audio files because the typical speech recognition system algorithms do not predict acoustic similarity (recognize speech) between a combination of a text form of an utterance and an audio file of an utterance by directly using the text form. Typical speech recognition system algorithms would convert text to speech, so that two audio files can be compared. Such conversion can cause speech recognition to be speech recognition system dependent. Therefore, a more reliable method to predict acoustic confusability is needed when using a combination of a text phrase and an audio file.
Using Acoustic Confusability Information When Developing Voice Application Software:
A voice user interface (i.e., a call flow) is developed or generated using text phrases representing voice commands. Acoustic confusability predictability information can, for example, be used when developing voice applications to avoid using a voice command in a call flow that may be confusable with other voice commands, for efficient, accurate and reliable call flow processing (voice command differentiation) by the voice application speech recognizer. However, typical speech recognizers compare sets of acoustic measures to classify sounds in spoken phrases, and do not directly use text phrases (i.e., not tied to comparing recorded speech signals) to predict acoustic confusability. Therefore, a process for development of a voice user interface typically includes counting syllables and comparison of vowels of the text phrases to select text phrases that are likely to be acoustically distinct (i.e., not confusable) when spoken as commands. To improve call flow processing using the typical acoustic confusability methods used by the speech recognizers would require, for example, text-to-speech conversions, which may not be efficient or practical. If acoustic confusability of text phrases representing voice commands can be predicated by directly using the text phrases, improved, more robust, and reliable voice user interfaces can be developed or generated.
Acoustic Distinctiveness of Spoken Phrases:
The acoustic distinctiveness of spoken phrases has been well understood for many years. The “landmark” theory of Stevens is one body of work that encapsulates much of what has been discovered about the acoustics of speech. Stevens, Kenneth, N., From Acoustic Cues To Segments, Features, and Words, Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000), Beijing China, Oct. 16–20, 2000—pp. 1–8; Stevens, K. N (1992) Lexical access from features, MIT Speech Communication Group Working Papers, VIII, 119–144; and Stevens, K. N., Manuel, S. Y., Shattuck-Hufnagel, S., and Liu, S. (1992), Implementation of a model for lexical access based on features, in J. J. Ohala, T. M. Nearey, G. L. Derwing, M. M. Hodge, and G. E. Wiebe (Eds.), Proceedings of the 1992 International Conference on Spoken Language Processing, Edmonton, Canada: University of Alberta—pp. 499–502 (hereinafter Stevens and the contents of which are hereby incorporated by reference).
Landmarks are points in a spoken phrase around which one may extract information about the underlying distinctive acoustic features (AFs). Landmarks mark perceptual foci and articulatory targets. One type of landmark is linked to glottal activity and can be used to identify vocalic segments of the speech signal. Other landmarks identify intervals of sonorancy, i.e. intervals when the oral cavity is relatively unconstricted. The most common landmarks are acoustically abrupt and are associated with consonantal segments, e.g., a stop closure and release.
Others have implemented methods to extend Stevens' landmark theory. Bitar, N., and Espy-Wilson, C. (1995), A signal representation of speech based on phonetic features, Proceedings of IEEE Dual-Use Technology and Applications Conference, 310–315 (hereinafter Bitar); and Automatic Detection of manner events based on temporal parameters, Proc. Eurospeech, September. '99, pp. 2797–2800 (hereinafter Salomon) (the contents of which are hereby incorporated by reference). Salomon has developed the following AFs as acoustic events: the manner-of-articulation or phonetic features (sonorant, syllabic, fricative, and consonantal) and the place-of-articulation phonetic or nonsyllabic features (labial, alveolar, and velar for stops; and palatal and alveolar for strident fricatives). Some acoustic events, such as the ones associated with the phonetic feature sonorant, segment the speech signal into regions. Others, such as those associated with nonsyllabic features, mark particular instants in time. From the Salomon AFs, the following twelve speech classes can be detected from the physical signal: syllabic vowel, syllabic nasal, syllabic liquid, semivowel, nasal, palatal fricative, alveolar fricative, affricate, labial stop, alveolar stop, velar stop, and weak fricative. A series of speech recognition experiments by Salomon illustrate robustness of the acoustic events based on AFs. Results disclosed in Salomon indicate that compared to traditional speech recognition processing, such as spectral processing with a state-of-the-art Hidden-Markov Model, the AFs can (a) better target the linguistic information in a speech signal, and (b) reduce inter-speaker variability. Therefore, the AFs can be used by voice applications when predicting acoustic confusability that is speaker-independent.