In this paper, a novel approach is proposed to automatically construct parallel discourse corpus for dialogue machine translation. Firstly, the parallel subtitle data and its corresponding monolingual movie script data are crawled and collected from Internet. Then tags such as speaker and discourse boundary from the script data are projected to its subtitle data via an information retrieval approach in order to map monolingual discourse to bilingual texts. We not only evaluate the mapping results, but also integrate speaker information into the translation. Experiments show our proposed method can achieve 81.79 and 98.64 accuracy on speaker and dialogue boundary annotation, and speaker-based language model adaptation can obtain around 0.5 BLEU points improvement in translation qualities. Finally, we publicly release around 100K parallel discourse data with manual speaker and dialogue boundary annotation.
There are two directions of work related to dialogue corpus construction. One is parallel corpora construction for dialogue or conversation MT @cite_21 @cite_10 @cite_12 @cite_17 @cite_6 @cite_3 @cite_24 . Thanks to the effects of crowdsourcing and fan translation in audiovisual translation @cite_22 , we can regard subtitles as parallel corpora. leveraged the existence of bilingual subtitles as a source of parallel data for the Chinese-English language pair to improve the MT systems in the movie domain. However, their work only considers sentence-level data instead of extracting more useful information for dialogues. Besides, Japanese researchers constructed a speech dialogue corpus for a machine interpretation system @cite_18 @cite_0 @cite_5 @cite_7 . They collected speech dialogue corpora for machine interpretation research via recording and transcribing Japanese English interpreters' consecutive simultaneous interpreting in the booth. The German VERBMOBIL speech-to-speech translation programme @cite_28 also collected and transcribed task-oriented dialogue data. This related work focused on speech-to-speech translation including three modules of automatic speech recognition (ASR), MT and text-to-speech(TTS).
@cite_18 @cite_22 @cite_7 @cite_28 @cite_21 @cite_6 @cite_3 @cite_24 @cite_0 @cite_5 @cite_10 @cite_12 @cite_17
2222731017 2552175949 2251796964 365755509
This paper proposes to use DTW to construct parallel corpora from difficult data. Parallel corpora are considered as raw material for machine translation (MT), frequently, MT systems use European or Canadian parliament corpora. In order to achieve a realistic machine translation system, we decided to use movie subtitles. These data could be considered difficult because they contain unfamiliar expressions, abbreviations, hesitations, words which do not exist in classical dictionaries (as vulgar words), etc. The obtained parallel corpora can constitute a rich ressource to train decoding spontaneous speech translation system. From 40 movies, we align 43013 English subtitles with 42306 French subtitles. This leads to 37625 aligned pairs with a precision of 92,3 . This paper proposes to use DTW to construct parallel corpora from difficult data. Parallel corpora are considered as raw material for machine translation (MT), frequently, MT systems use European or Canadian parliament corpora. In order to achieve a realistic machine translation system, we decided to use movie subtitles. These data could be considered difficult because they contain unfamiliar expressions, abbreviations, hesitations, words which do not exist in classical dictionaries (as vulgar words), etc. The obtained parallel corpora can constitute a rich ressource to train decoding spontaneous speech translation system. From 40 movies, we align 43013 English subtitles with 42306 French subtitles. This leads to 37625 aligned pairs with a precision of 92,3 . Corpus-based approaches to machine translation (MT) rely on the availability of parallel corpora. To produce user-acceptable translation outputs, such systems need high quality data to be efficiently trained, optimized and evaluated. However, building high quality dataset is a relatively expensive task. In this paper, we describe the data collection and analysis of a large database of 10.881 SMT translation output hypotheses manually corrected. These post-editions were collected using Amazon’s Mechanical Turk, following some ethical guidelines. A complete analysis of the collected data pointed out a high quality of the corrections with more than 87 of the collected post-editions that improve hypotheses and more than 94 of the crowdsourced post-editions which are at least of professional quality. We also post-edited 1,500 gold-standard reference translations (of bilingual parallel corpora generated by professional) and noticed that 72 of these translations needed to be corrected during post-edition. We computed a proximity measure between the different kind of translations and pointed out that reference translations are as far from the hypotheses as from the corrected hypotheses (i.e. the post-editions). In light of these last findings, we discuss the adequation of text-based generated reference translations to train setence-to-sentence based SMT systems. Speech-to-speech translation has been an important research topic with the advance of technologies for speech processing and language processing. This paper describes a bilingual speech dialogue corpus which has been constructed for research on simultaneous machine interpretation at the Center for Integrated Acoustic Information Research (CIAIR), Nagoya University. The corpus has been implemented by collecting simulated cross-lingual conversations between English speech and Japanese speech through simultaneous interpretation, and by transcribing them manually with bilingual sentence alignment. In the year 2002, 216 spoken dialogues have been collected under a real environment, and transcribed into text files consisting of about 300,000 morphemes. In order to utilize the bilingual corpus effectively, every source utterance speech has been segmented into interpreting units according to its word-for-word translation and the word alignment of them. The interpreting unit means a linguistic chunk that could be interpreted separately and simultaneously. This paper has investigated linguistic characters of such the unit, and examined the feasibility of simultaneous machine interpretation.