TITLE: A Vietnamese Dataset for Evaluating Machine Reading Comprehension 

ABSTRACT: Over 97 million people speak Vietnamese as their native language in the
world. However, there are few research studies on machine reading comprehension
(MRC) for Vietnamese, the task of understanding a text and answering questions
related to it. Due to the lack of benchmark datasets for Vietnamese, we present
the Vietnamese Question Answering Dataset (UIT-ViQuAD), a new dataset for the
low-resource language as Vietnamese to evaluate MRC models. This dataset
comprises over 23,000 human-generated question-answer pairs based on 5,109
passages of 174 Vietnamese articles from Wikipedia. In particular, we propose a
new process of dataset creation for Vietnamese MRC. Our in-depth analyses
illustrate that our dataset requires abilities beyond simple reasoning like
word matching and demands single-sentence and multiple-sentence inferences.
Besides, we conduct experiments on state-of-the-art MRC methods for English and
Chinese as the first experimental models on UIT-ViQuAD. We also estimate human
performance on the dataset and compare it to the experimental results of
powerful machine learning models. As a result, the substantial differences
between human performance and the best model performance on the dataset
indicate that improvements can be made on UIT-ViQuAD in future research. Our
dataset is freely available on our website to encourage the research community
to overcome challenges in Vietnamese MRC.
Over 97 million people speak Vietnamese as their native language in the
world. However, there are few research studies on machine reading comprehension
(MRC) for Vietnamese, the task of understanding a text and answering questions
related to it. Due to the lack of benchmark datasets for Vietnamese, we present
the Vietnamese Question Answering Dataset (UIT-ViQuAD), a new dataset for the
low-resource language as Vietnamese to evaluate MRC models. This dataset
comprises over 23,000 human-generated question-answer pairs based on 5,109
passages of 174 Vietnamese articles from Wikipedia. In particular, we propose a
new process of dataset creation for Vietnamese MRC. Our in-depth analyses
illustrate that our dataset requires abilities beyond simple reasoning like
word matching and demands single-sentence and multiple-sentence inferences.
Besides, we conduct experiments on state-of-the-art MRC methods for English and
Chinese as the first experimental models on UIT-ViQuAD. We also estimate human
performance on the dataset and compare it to the experimental results of
powerful machine learning models. As a result, the substantial differences
between human performance and the best model performance on the dataset
indicate that improvements can be made on UIT-ViQuAD in future research. Our
dataset is freely available on our website to encourage the research community
to overcome challenges in Vietnamese MRC.
A Vietnamese Dataset for Evaluating Machine Reading Comprehension
http://arxiv.org/abs/2009.14725v3