Machine translation for Arabic dialects (survey) <s> 11 <s> Apparatus for forming a barrier on a support surface to restrict movement of a baby walker. The apparatus includes at least one elongated barrier element positionable on the support surface which is rotatable when engaged by a baby walker wheel to bring the elongated barrier element into contact with the walker body and prevent further movement of the baby walker. <s> BIB001 </s> Machine translation for Arabic dialects (survey) <s> 11 <s> Abstract : Errors in machine translations of English-Iraqi Arabic dialogues were analyzed at two different points in the systems' development using Human Translation Error Rate (HTER) methods to identify errors and human annotations to refine Translation Error Rate (TER) annotations. Although the frequencies of errors in the more mature systems were lower, the proportions of error types exhibited little change. Results include high frequencies of pronoun errors in translations to English, high frequencies of subject person inflection in translations to Iraqi Arabic, similar frequencies of word order errors in both translation directions, and very low frequencies of polarity errors. The problems with many errors can be generalized as the need to insert lexemes not present in the source or vice versa, which includes errors in multi-word expressions. Discourse context will be required to resolve some problems with deictic elements like pronouns. <s> BIB002 </s> Machine translation for Arabic dialects (survey) <s> 11 <s> A major challenge for statistical machine translation (SMT) of Arabic-to-English user-generated text is the prevalence of text written in Arabizi, or Romanized Arabic. When facing such texts, a translation system trained on conventional Arabic-English data will suffer from extremely low model coverage. In addition, Arabizi is not regulated by any official standardization and therefore highly ambiguous, which prevents rule-based approaches from achieving good translation results. In this paper, we improve Arabizi-to-English machine translation by presenting a simple but effective Arabizi-to-Arabic transliteration pipeline that does not require knowledge by experts or native Arabic speakers. We incorporate this pipeline into a phrase-based SMT system, and show that translation quality after automatically transliterating Arabizi to Arabic yields results that are comparable to those achieved after human transliteration. <s> BIB003
The performance of the system were improved by considering and treating the different dialects as different domains. An example of adaptation technique is using instance weighting of translation models to improve the translation quality by giving more weights to Egyptian than MSA or other Arabic dialects. It should be noted that the systems were adapted by using data selection techniques because the training data include various genres (News, Web, Discussion forums, SMS/CHAT). Data selection techniques consist of selecting the relevant sentences from monolingual corpora to improve and adapt the language models, or selecting the most relevant sentences from the bilingual corpora to improve the translation models. Another possible way of improving the system performance and translation quality was morphological segmentation. Several segmentation schemes were evaluated. Furthermore, in order to deal with the out-of-vocabulary words and to decrease the OOV rate proper noun transliteration was performed. Recently, as regards the script used in dialectal texts, a new research line has been open up for Arabic dialect MT. It concerns Arabizi, also known as Romanized Arabic or Arabish. Arabizi is a non-standard writing system that uses Latin characters 8 to write Arabic dialects. It is widely used in the context of social media communications like Facebook, Twitter and YouTube, chat rooms and SMS. Arabizi is a mixture of both transliteration and transcription mappings, it does not obey to strict rules, it differs from one dialect to another, even in the same dialect community it differs from one user to another. Despite it has no standard form, a large amount of Arabizi data is generated by everyday communication (social media, SMS, etc). Thus, Arabizi creates new needs in the area of dialect NLP, it brings new challenges, especially for Machine Translation. It should be noted that the NIST OpenMT15 9 evaluation competition focused on informal data genres (SMS/Chat and Conversational Telephone Speech (CTS)) in Arabic dialect, precisely Egyptian, and Mandarin Chinese. 10 The task consisted in translating from Egyptian dialect and Mandarin Chinese into English. 11 It is worth noting that Egyptian dialect data within this campaign is a mixture of texts 8 Including letters and numbers 9 Open Machine Translation 2015 10 https://www.nist.gov/sites/default/files/documents/itl/iad/mig/ OpenMT15_EvalPlan_v0-9.pdf In this study, the authors created a deromanization module (converts Arabizi to Arabic script) whose output is translated into English via a SMT system trained on informal Arabic/English parallel and monolingual data (from DARPA BOLT). Their deromanization approach uses a character-based weighted finite state transducers (wFSTs) Mohri (1997) with a 5-gram character-based language model of Arabic dialect (learned from 5.4M words). We note that a character-based language model is used instead of a word-based one to avoid OOV words. Three methods were experimented to build Arabizi-to-Arabic script wFST, (1) manually by human experts 12 , (2) automatically by using machine translation and (3) hybrid method (combining the two last). The first method consists in asking a native Arabic speaker to generate probabilistic character sequence pairs in order to encode the wFST transitions, whereas the automatic method is a SMT system trained on a corpus of 863 Arabizi/Arabic dialect (Arabic script) word pairs (where the words pairs are 12 Familiar with finite-state machines 13 treated as sentence pairs and character are treated as words). According to the authors, this method produces more correspondences than the manual method and sequence pairs with longer context but generates also a set of noisy pairs that are useless. Another negative aspect of this method is that it does not generate vowel-dropping sequence pairs (that are taken into account by the first method). The hybrid method involves using sequences pairs (with Arabizi length of less than three characters) from those generated by the SMT system in addition to vowel-dropping sequence pairs from the manual wFST pairs. For the evaluation of both the deromanization module and the Arabizi-English SMT, the authors used two parallel corpora of Arabizi-English of 7,794 and 27,901 aligned sentences with reference deromanizations of the Arabizi side of each corpus. For the deromanization module, the automatic and hybrid methods outperform the manual one. However, the results of the hybrid approach are slightly better than the automatic approach. As regards the SMT systems scores, they track those of deromanization results. The SMT using automatically learned wFST approach outperforms the manual wFST (BLEU scores of respectively 12.0 and 8.9 Vs 15.1 and 13.2). In addition, the BLEU score (15.3 and 13.4) of the SMT system using the hybrid approach outperforms slightly the score of the SMT system using the automatic approach (15.1 and 13.2). BIB003 attempted to improve Arabizi-to-English machine translation by using an Arabizi-to-Arabic script converter that does not require human knowledge (experts or native Arabic speakers). This converter has been incorporated into a phrase-based SMT system whose performance yields results that are comparable to those achieved after human transliteration. This work uses a set of resources including : a large Arabic dialect-English parallel corpus (1.75M sentence pairs with 52.9M Arabic tokens), a small tri-text Arabizi-dialect (Arabic script)-English (10K parallel sentences belonging to the SMS and chat genres 13 ) from which 1788 parallel Arabizi-dialect (Arabic script) sentences were split into two test sets for evaluation, and finally, an Arabizi-English parallel corpus 14 crawled from a variety of web pages (10K sentence pairs with 180K Arabizi tokens). The first step of deromanization is generating transliteration candidates, this is done by character mapping module 15 following the phrase-based SMT paradigm. Since the generated candidates could include character sequences that are not actual Arabic words, they are filtered by comparing them to a large Arabic dialect vocabulary (200K of distinct words) and the OOV candidates are then eliminated. This operation reduced the number of candidates for a given Arabizi word by 50% and also excluded Arabizi words with character repetitions. 16 After generating candidates and filtering steps, an ambiguous Arabizi-to-dialect(Arabic script) lexicon is created. This lexicon, in addition to a 3-gram Arabic dialect language model (trained on the source side of the available parallel dialect (Arabic script)-English corpora) are passed through a contextual disambiguation process using srilm-disambig 17 in order to search for the best transliteration of each Arabizi sentence. At this stage (we call it a first variant of the romanization), the WERs (Word Error Rates) recorded for the two set tests were 46.4% and 50.8%. For improving these results, the authors exploited transliterated word pairs extracted from the tritext Arabizi-dialect (Arabic script)-English described above. They added them to the transliterated lexicon used by the contextual disambiguation by prioritizing them with a high score (0.9 Vs 0.1 for the other transliteration candidates). This step (the second variant of deromanization) contributed to an improvement of the WERs by 50% (25.7% and 027.9%) on the two test sets. This transliteration module has been incorporated into an in-house phrase-based SMT trained on the collection of dialect (Arabic script)-English corpora described above (1.75M parallel sentences with 52.9M Arabic tokens) and a 5-gram English language model. On the other hand, the Arabizi-English corpus of web-crawled user comments has been used to train a small SMT system whose phrase translation and phrase reordering models have been merged to the main SMT system models. This increases the chance of translating (directly by the Arabizi-English models) a non-transliterated Arabizi word. For the two variants of the transliteration module, the SMT system has been evaluated using BLEU score. The best BLEU is recorded for the transliteration module that uses character-level mapping with contextual disambiguation augmented by words pairs (second variant) with 8.68 and 10.32 on the two test sets Vs a BLEU of 7.46 and 9.42 (for the first variant). Table 2 provides a summary of MT work listed above with regard to concerned dialects, translation direction and pivoting through MSA. MuDMaT. Another project dedicated to machine translation of Arabic dialects is MuDMAT project (Multi-Dialect Machine Translation)  supported by NSERC. 18 MuDMaT is speared over the period of (2014-2017). It aims to build MT systems between Maghreb dialects (Algerian, Moroccan and Tunisian), MSA and French using hybrid approach. According to the author, a demonstration of a rule-based machine translation from Tunisian dialect to MSA and French was achieved. All the work cited above is related to text machine translation. For speech translation there are no relevant projects dedicated for Arabic dialects, except those funded by DARPA such as TRANSTAC 19 project  , a predecessor program to BOLT which deals with MT between Iraqi dialect and English. The goal of TRANSTAC is a rapid development of bi-directional translation systems that allow speakers of different languages to communicate in real-world tactical situations. Several prototype systems were developed for military and medical screening domains to enable conversations with local foreign language speakers of Iraqi Arabic, Mandarin, Farsi, Pashto, and Thai. Some research was dedicated to evaluate MT scores of Iraqi Arabic and English translators such as BIB002 and BIB001 . In the same context, IBM MASTOR  , is a speech-to-speech translation system that translates spontaneous free-form speech in real-time on both laptop and hand-held PDAs for two language pairs, English-Mandarin Chinese, and English-Arabic dialect.