Review on automatic speech recognition quality estimation (ASR QE).The application of ASR QE in ASR system combination for both single-microphone multiple-ASR system task and on multiple-microphone multiple-ASR system task.Ranking the system combination inputs based on predicted quality.Management of tied ranks.Automatically finding the optimum level of combination for each segment. Recognizer Output Voting Error Reduction (ROVER) has been widely used for system combination in automatic speech recognition (ASR). In order to select the most appropriate words to insert at each position in the output transcriptions, some ROVER extensions rely on critical information such as confidence scores and other ASR decoder features. This information, which is not always available, highly depends on the decoding process and sometimes tends to overestimate the real quality of the recognized words. In this paper we propose a novel variant of ROVER that takes advantage of ASR quality estimation (QE) for ranking the transcriptions at segment level instead of: i) relying on confidence scores, or ii) feeding ROVER with randomly ordered hypotheses. We first introduce an effective set of features to compensate for the absence of ASR decoder information. Then, we apply QE techniques to perform accurate hypothesis ranking at segment-level before starting the fusion process. The evaluation is carried out on two different tasks, in which we respectively combine hypotheses coming from independent ASR systems and multi-microphone recordings. In both tasks, it is assumed that the ASR decoder information is not available. The proposed approach significantly outperforms standard ROVER and it is competitive with two strong oracles that exploit prior knowledge about the real quality of the hypotheses to be combined. Compared to standard ROVER, the absolute WER improvements in the two evaluation scenarios range from 0.5 to 7.3 .
An alternative to signal-level combination and best microphone selection is represented by hypothesis-level combination. Along this direction, @cite_31 proposes to use CNC for lecture transcriptions and @cite_17 extends it with a hybrid approach that leverages beamforming and signal-level diversity to transcribe meetings. In @cite_54 , inter-microphone agreement is used to build a confusion network from multiple lattices and improve ASR in a domotic application.
@cite_31 @cite_54 @cite_17
2134533334 1532798076 2290469434 2099984896
The use of multiple distant microphones has been widely studied for meeting recognition. The two most widely used approaches are 1) combination at the signal level, via blind beamforming, followed by recognition of a single enhanced audio signal, and 2) independent, logically parallel recognition of the multiple audio sources followed by hypothesis-level combination. In this paper we investigate how these two approaches compare for state-of-the-art recognition systems applied to meeting data from the two most recent NIST Rich Transcription evaluations. Our results show that beamforming is the superior approach, giving more accurate results while being inherently less computationally demanding. We then propose a hybrid approach that leverages both beamforming and signal-level diversity for systemcombination, and showthat this approach gives gains over either of the old methods. A multi-microphone hypothesis combination approach, suitable for the distant-talking scenario, is presented in this paper. The method is based on the inter-microphone agreement of information, extracted at speech recognition level. Particularly, temporal information is exploited to organize the clusters that shape the resulting confusion network, and to reduce the global hypothesis search space. As a result, a single combined confusion network is generated from multiple lattices. The approach offers a novel perspective to solutions based on confusion network combination. The method was evaluated in a simulated domestic environment equipped with largely spaced microphones. The experimental evidence suggests that results, comparable or, in some cases, better than the state of the art, can be achieved under optimal configurations with the proposed method. Speech recognition in a realistic noisy environment using multiple microphones is the focal point of the third CHiME challenge. Over the baseline ASR system provided for this challenge, we apply state of the art algorithms for boosting acoustic model learning and hypothesis rescoring to improve the final output. To this aim, we first use the automatic transcription of each channel to re-train the acoustic model for that channel and then we apply linear language model rescoring to find a better solution in the n-best list. LM rescoring is performed using an efficient set of N-gram and Recurrent Neural Network LM (RNNLM) trained on a wisely-selected text set. In the experiments, we show that the proposed approach improves not only the individual channel transcription, but also the enhanced channels produced by MVDR and delay-and-sum beamforming. This paper proposes a new robust adaptive beamformer applicable to microphone arrays. The proposed beamformer is a generalized sidelobe canceller (GSC) with a new adaptive blocking matrix using coefficient-constrained adaptive filters (CCAFs) and a multiple-input canceller with norm-constrained adaptive filters (NCAFs). The CCAFs minimize leakage of the target-signal into the interference path of the GSC. Each coefficient of the CCAFs is constrained to avoid mistracking. The input signal to all the CCAFs is the output of a fixed beamformer. In the multiple-input canceller, the NCAFs prevent undesirable target-signal cancellation when the target-signal minimization at the blocking matrix is incomplete. The proposed beamformer is shown to be robust to target-direction errors as large as 200 with almost no degradation in interference-reduction performance, and it can be implemented with several microphones. The maximum allowable target-direction error can be specified by the user. Simulated anechoic experiments demonstrate that the proposed beamformer cancels interference by over 30 dB. Simulation with real acoustic data captured in a room with 0.3-s reverberation time shows that the noise is suppressed by 19 dB. In subjective evaluation, the proposed beamformer obtains 3.8 on a five-point mean opinion score scale, which is 1.0 point higher than the conventional robust beamformer.