The invention relates to a method and a configuration for producing a sound.
Such a configuration and such a method are known from R. E. Donovan et al.: xe2x80x9cAutomatic Speech Synthesiser Parameter Estimation using HMMsxe2x80x9d, IEEE 1995, pages 640-643; hereinafter xe2x80x9cDonovan et al.xe2x80x9d This publication discloses the production of a decision tree, for its part permitting a cluster formation for the modeling of triphones. For this purpose, a series of questions, which relate directly to a phonetic context and can be answered with yes or no, is used. Each answer sets up a further subtree. Consequently, training data of naturally spoken speech are projected onto the branches and finally leaves of the decision tree.
The decision tree is used for calculating the leaves to be used in order to determine hidden Markov models for all possible triphones not covered by the training data.
Hidden Markov models (HMM) for the modeling of sounds are known from E. G. Schukat-Talamazzini: Automatische Spracherkennung xe2x80x94Grundlagen, statistische Modelle und effiziente Algorithmen [Automatic Speech Recognitionxe2x80x94Principles, Statistical Models And Efficient Algorithms], Vieweg and Sohn Verlagsgesellschaft mbH, Brunswick/Wiesbaden 1995, pages 125-139. In the linguistic production of a word, the constituent sounds are realized with variable duration and in varying spectral composition. Dependent on the rate and rhythm of the speech, each individual phonetic segment of the utterance is allotted an unpredictable number of feature vectors; each vector includes not only its phonetic content but also information components relating to the speaker, ambience and slurring, which make identification of the sounds significantly more difficult.
These conditions can be modeled in a simplified form by a two-stage process, as FIG. 1 shows by the example of the word xe2x80x9chabenxe2x80x9d [have]. For the phonemes of the word, a corresponding number of states 102 to 106 are reserved in the model and are run through along the direction of the arrow 101 for producing speech. At every time pulse, it is possible to remain in the current state or transfer to the succeeding state; the system behaves randomly and is determined by the transfer probabilities 107 to 111 depicted. For example, the state 103 belonging to the phoneme /a/ is adopted over a number of successive brief analysis intervals (on average over ten), whereas realizations of the plosive /b/ take less time.
While the first stage described of the random process models the time distortion of different pronunciation variants, a second stage serves for sensing spectral variations. Linked to each state of the word model is a statistical output function that weights the phonetic realization alternatives. In the example of FIG. 1, for the production of the phoneme /a/ not only the actually matching phone class 113 but also the phone class 114 is permitted with a positive probability (here: 0.1). The phone class 118 is likewise permitted for the production of the phoneme /n/ with a probability of 0.3. The formalism described also allows a description of an optional sound elimination, expressed by the xe2x80x9cbridgingxe2x80x9d 119 of the state 105 by a direct transfer between the states 104 and 106. The bridging is given a probability of 0.2 by way of example.
The transfer probabilities of the hidden Markov model can be determined based on training data. When it has been fully trained, the HMM represents a blueprint for the production of sequences of sounds (cf. Schukat-Talamazzini, pages 127-139). One method of training the HMM is to use the Baum-Welch algorithm.
However, in the method described in Donovan et al. it is disadvantageous that, in the decision tree, only the leaves respectively found are used for the sound modeling.
It is accordingly an object of the invention to provide a method and configuration for determining a representative sound, method for synthesizing speech, and method for speech processing that overcomes the hereinafore-mentioned disadvantages of the heretofore-known devices of this general type and that, when determining a representative sound (from a large number of sounds), accounts for not only a structure, devised according to predetermined criteria, but also a characteristic state criterion of this structure.
With the foregoing and other objects in view, there is provided, in accordance with the invention, a method for determining a representative sound based on a structure. The first step of the method is forming, from a sound, a structure having a characteristic state criterion. The next step is providing a set of sound models, each sound model having a representative with a plurality of quality criterion. The next step is determining, in the structure, a first sound model from the set of sound models matching a first quality criterion. The next step is determining a second sound model from the set of sound models depending on the characteristic state criterion of the structure. The next step is forming an overall quality criterion for each representative by assessing representatives of the first and the second sound model with regard to a second quality criterion in addition to the first quality criterion. The next step is determining a representative having an adequate overall quality criterion with regard to the first and second quality criteria as a representative sound.
In the method for determining a representative sound based on a structure that includes a set of sound models, each sound model has at least one representative for the modeled sound. In the structure, a first sound model, matching with regard to a first quality criterion, is determined from the set of sound models. Dependent on a characteristic state criterion of the structure, at least one second sound model is determined from the set of sound models. Representatives of the first sound model and of the at least one second sound model are assessed in addition to the first quality criterion with regard to a second quality criterion. From the representatives of the first and the at least one second sound model, that at least one representative which has an adequate overall quality criterion with regard to the first and second quality criteria is determined as a representative sound.
In accordance with how the structure is ordered, a search is conducted within the structure for a matching sound model for the sound to be produced. In this case, xe2x80x9cmatchingxe2x80x9d applies with regard to the first quality criterion, which is predetermined in particular by the structure.
The structure may be configured as a tree structure, preferably as a binary tree. Such a tree structure has nodes (for the embodiment of the sound models), branches (for the hierarchical subdivision of the sound models dependent on the criteria on the basis of which the structure is constructed) and leaves (nodes from which no further branch extends).
The structure constructed based on predetermined criteria is then used in order to determine, depending on the characteristic state of the structure (in particular the tree structure) at least one second sound model from the set of sound models. In this case, the characteristic state criterion in the structure may be a measure of distance from the first sound model. In the case of the binary tree as the structure, all the sound models within a predetermined distance from the first sound model may be regarded as second sound models. Here, the term xe2x80x9cdistancexe2x80x9d is not necessarily to be interpreted in the local sense; rather, the xe2x80x9cdistancexe2x80x9d may also concern a dimension of distance with regard to one or more predetermined criteria.
With the first sound model and a set of second sound models that satisfy the characteristic state criterion and have the predetermined distance from the first sound model, a second quality criterion is determined for the representatives of the sound models. The overall quality criterion for each representative is made up of the first and the at least one second quality criteria. A candidate among the representatives that is suitable for producing the sound is determined by its xe2x80x9cadequatexe2x80x9d overall quality criterion that is an overall quality criterion that has a predetermined minimum value.
A development of this is that the overall quality criterion is an arithmetic combination of the quality criteria.
In particular, it is a development that a representative of the at least one second sound model is additionally provided with a reduced quality, resulting from the characteristic state criterion, in comparison with a representative of the first sound model. Here, the distance mentioned above of the at least one second sound model from the first sound model is preferably used: a great distance indicates a great deviation from the first sound model, originally found as matching according to the structure. For such a great distance, a kind of xe2x80x9cpenaltyxe2x80x9d is awarded for the representative of the at least one second sound model, this penalty having an effect on the overall quality criterion.
One refinement is that a best representative, with regard to the overall quality criterion, is determined as the representative sound.
Another refinement is that the structure is devised based on at least one of the following criteria:
a) linguistic criteria, for example grammatical properties of a word;
b) articulatory criteria, in particular the location of the articulation; and
c) phonetic criteria, for example classified on the basis of plosive or nasal sounds.
An additional development is that the sound may be formed as one of the following possibilities:
a) phoneme,
b) biphone,
c) triphone,
d) syllable,
e) combination of syllables,
f) word,
g) combination of words.
It is also a refinement that, whenever the sound represents a combination of sounds, a construction of the structure takes place on the basis of context criteria of the sound within the combination of sounds. This means in particular that, in the case of a combination of sounds, questioning of the kind
xe2x80x9cIs the sound to the left of the current sound a vowel?xe2x80x9d
or
xe2x80x9cIs the sound to the left of the current sound a plosive?xe2x80x9d
can in each case be answered with YES or NO and consequently the context to the right or left is used for constructing the structure, in particular the binary tree.
Within an additional refinement, the sound model takes the form of a hidden Markov model (HMM).
A development is that a selection of n representatives which, based on their overall quality criterion, are in each case best suited for determining the representative sound is represented. The number n can in this case be predetermined application-dependently; the best representatives are preferably presented in the form of a list.
Furthermore, it is a refinement that the first sound model is determined for the determination of the representative sound as soon as no improvement of the first quality criterion can be achieved any longer by a further search in the structurexe2x80x94in particular the tree structure.
Another refinement is that the first sound model is determined as soon as fewer representatives than are required for the determination of the representative sound are available with respect to a sound model from the set of sound models with a possibly better first quality criterion. In other words, in the search for an adequately good first sound model, only those sound models that comprise more representatives than is predetermined by a threshold value are taken into account.
Within a further refinement, a fuzzy logic assessment is performed for each second quality criterion. As a result, it is ensured that for each second quality criterion an own assessment is performed, carried out in particular in a normalized manner with regard to comparability with other assessments. Fuzzy logic provides a suitable method for such a normalized assessment.
In accordance with a further object of the invention, the method described above can be used in speech processing, in particular in speech synthesis.
Also provided for achieving the object is a configuration for determining a representative sound based on a structure that comprises a set of sound models. Each sound model has at least one representative for the modeled sound. The configuration has a processor unit that is set up in such a way that
a) in the structure, a first sound model, matching with regard to a first quality criterion, can be determined from the set of sound models;
b) dependent on a characteristic state criterion of the structure, at least one second sound model can be determined from the set of sound models;
c) representatives of the first and the at least one second sound model can be assessed in addition to the first quality criterion with regard to a second quality criterion;
d) that at least one representative which has an adequate overall quality criterion with regard to the first and second quality criteria can be determined as a representative sound.
This configuration is suitable in particular for carrying out the method according to the invention or one of its developments explained above.
Other features that are considered as characteristic for the invention are set forth in the appended claims.
Although the invention is illustrated and described herein as embodied in a method and configuration for determining a representative sound, method for synthesizing speech, and method for speech processing, it is nevertheless not intended to be limited to the details shown, since various modifications and structural changes may be made therein without departing from the spirit of the invention and within the scope and range of equivalents of the claims.
The construction and method of operation of the invention, however, together with additional objects and advantages thereof will be best understood from the following description of specific embodiments when read in connection with the accompanying drawings.