TITLE: The first large scale collection of diverse Hausa language datasets 

ABSTRACT: Hausa language belongs to the Afroasiatic phylum, and with more
first-language speakers than any other sub-Saharan African language. With a
majority of its speakers residing in the Northern and Southern areas of Nigeria
and the Republic of Niger, respectively, it is estimated that over 100 million
people speak the language. Hence, making it one of the most spoken Chadic
language. While Hausa is considered well-studied and documented language among
the sub-Saharan African languages, it is viewed as a low resource language from
the perspective of natural language processing (NLP) due to limited resources
to utilise in NLP-related tasks. This is common to most languages in Africa;
thus, it is crucial to enrich such languages with resources that will support
and speed the pace of conducting various downstream tasks to meet the demand of
the modern society. While there exist useful datasets, notably from news sites
and religious texts, more diversity is needed in the corpus.
  We provide an expansive collection of curated datasets consisting of both
formal and informal forms of the language from refutable websites and online
social media networks, respectively. The collection is large and more diverse
than the existing corpora by providing the first and largest set of Hausa
social media data posts to capture the peculiarities in the language. The
collection also consists of a parallel dataset, which can be used for tasks
such as machine translation with applications in areas such as the detection of
spurious or inciteful online content. We describe the curation process -- from
the collection, preprocessing and how to obtain the data -- and proffer some
research problems that could be addressed using the data.
Hausa language belongs to the Afroasiatic phylum, and with more
first-language speakers than any other sub-Saharan African language. With a
majority of its speakers residing in the Northern and Southern areas of Nigeria
and the Republic of Niger, respectively, it is estimated that over 100 million
people speak the language. Hence, making it one of the most spoken Chadic
language. While Hausa is considered well-studied and documented language among
the sub-Saharan African languages, it is viewed as a low resource language from
the perspective of natural language processing (NLP) due to limited resources
to utilise in NLP-related tasks. This is common to most languages in Africa;
thus, it is crucial to enrich such languages with resources that will support
and speed the pace of conducting various downstream tasks to meet the demand of
the modern society. While there exist useful datasets, notably from news sites
and religious texts, more diversity is needed in the corpus.
  We provide an expansive collection of curated datasets consisting of both
formal and informal forms of the language from refutable websites and online
social media networks, respectively. The collection is large and more diverse
than the existing corpora by providing the first and largest set of Hausa
social media data posts to capture the peculiarities in the language. The
collection also consists of a parallel dataset, which can be used for tasks
such as machine translation with applications in areas such as the detection of
spurious or inciteful online content. We describe the curation process -- from
the collection, preprocessing and how to obtain the data -- and proffer some
research problems that could be addressed using the data.
The first large scale collection of diverse Hausa language datasets
http://arxiv.org/abs/2102.06991v2