Statistical phrase-based translation learns translation rules from bilingual corpora , and has traditionally only used monolingual evidence to construct features that rescore existing translation candidates . In this work , we present a semi-supervised graph-based approach for generating new translation rules that leverages bilingual and monolingual data . The proposed technique first constructs phrase graphs using both source and target language monolingual corpora . Next , graph propagation identifies translations of phrases that were not observed in the bilingual corpus , assuming that similar phrases have similar translations . We report results on a large Arabic-English system and a medium-sized Urdu-English system . Our proposed approach significantly improves the performance of competitive phrasebased systems , leading to consistent improvements between 1 and 4 BLEU points on standard evaluation sets . Statistical approaches to machine translation ( SMT ) use sentence-aligned , parallel corpora to learn translation rules along with their probabilities . With large amounts of data , phrase-based translation systems ( Koehn et al . , 2003 ; Chiang , 2007 ) achieve state-of-the-art results in many typologically diverse language pairs ( Bojar et al . , 2013 ) . However , the limiting factor in the success of these techniques is parallel data availability . Even in resource-rich languages , learning reliable translations of multiword phrases is a challenge , and an adequate phrasal inventory is crucial ⇤ This work was done while the first author was interning at Microsoft Research for effective translation . This problem is exacerbated in the many language pairs for which parallel resources are either limited or nonexistent . While parallel data is generally scarce , monolingual resources exist in abundance and are being created at accelerating rates . Can we use monolingual data to augment the phrasal translations acquired from parallel data ? The challenge of learning translations from monolingual data is of long standing interest , and has been approached in several ways ( Rapp , 1995 ; Callison-Burch et al . , 2006 ; Haghighi et al . , 2008 ; Ravi and Knight , 2011 ) . Our work introduces a new take on the problem using graphbased semi-supervised learning to acquire translation rules and probabilities by leveraging both monolingual and parallel data resources . On the source side , labeled phrases ( those with known translations ) are extracted from bilingual corpora , and unlabeled phrases are extracted from monolingual corpora ; together they are embedded as nodes in a graph , with the monolingual data determining edge strengths between nodes ( § 2.2 ) . Unlike previous work ( Irvine and Callison-Burch , 2013a ; Razmara et al . , 2013 ) , we use higher order n-grams instead of restricting to unigrams , since our approach goes beyond OOV mitigation and can enrich the entire translation model by using evidence from monolingual text . This enhancement alone results in an improvement of almost 1.4 BLEU points . On the target side , phrases initially consisting of translations from the parallel data are selectively expanded with generated candidates ( § 2.1 ) , and are embedded in a target graph . We then limit the set of translation options for each unlabeled source phrase ( § 2.3 ) , and using a structured graph propagation algorithm , where translation information is propagated from labeled to unlabeled phrases proportional to both source and target phrase similarities , we estimate probability distributions over translations for Figure 1 : Example source and target graphs used in our approach . Labeled phrases on the source side are black ( with their corresponding translations on the target side also black ) ; unlabeled and generated ( § 2.1 ) phrases on the source and target sides respectively are white . Labeled phrases also have conditional probability distributions defined over target phrases , which are extracted from the parallel corpora . the unlabeled source phrases ( § 2.4 ) . The additional phrases are incorporated in the SMT system through a secondary phrase table ( § 2.5 ) . We evaluated the proposed approach on both Arabic-English and Urdu-English under a range of scenarios ( § 3 ) , varying the amount and type of monolingual corpora used , and obtained improvements between 1 and 4 BLEU points , even when using very large language models . In this work , we presented an approach that can expand a translation model extracted from a sentence-aligned , bilingual corpus using a large amount of unstructured , monolingual data in both source and target languages , which leads to improvements of 1.4 and 1.2 BLEU points over strong baselines on evaluation sets , and in some scenarios gains in excess of 4 BLEU points . In the future , we plan to estimate the graph structure through other learned , distributed representations .
The performance of statistical phrase-based translation is limited by the size of the available phrasal inventory both for resource rich and poor language pairs.
They propose a semi-supervised approach that produces new translation rules from monolingual data by phrase graph construction and graph propagation techniques.
Their method significantly improves over existing phrase-based methods on Arabic-English and Urdu-English systems when large language models are used.