Asaad Alghamdi, Xinyu Duan, Wei Jiang, Zhenhai Wang, Yimeng Wu, Qingrong Xia, Zhefeng Wang, Yi Zheng, Mehdi Rezagholizadeh, Baoxing Huai, Peilun Cheng, Abbas Ghaddar &ndash; AI Cognitive Team, Tonomus; Huawei Cloud Computing Technologies Co., Ltd.; Huawei Technologies Co., Ltd.
AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing
We mainly leverage all (up to July 2022) of the 90 Common Crawl³ monthly web scrapes in order to collect massive amount of Arabic textual data. [...] Our pre-training corpus is mainly sourced from the publicly available web scrapes of the Common Crawl (CC) project. We downloaded 90 shards of CC monthly data ranging from May 2013 (the earliest available) up to July 2022. Also, we use [...]