TITLE: ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning 

ABSTRACT: Understanding how events are semantically related to each other is the
essence of reading comprehension. Recent event-centric reading comprehension
datasets focus mostly on event arguments or temporal relations. While these
tasks partially evaluate machines' ability of narrative understanding,
human-like reading comprehension requires the capability to process event-based
information beyond arguments and temporal reasoning. For example, to understand
causality between events, we need to infer motivation or purpose; to establish
event hierarchy, we need to understand the composition of events. To facilitate
these tasks, we introduce ESTER, a comprehensive machine reading comprehension
(MRC) dataset for Event Semantic Relation Reasoning. The dataset leverages
natural language queries to reason about the five most common event semantic
relations, provides more than 6K questions and captures 10.1K event relation
pairs. Experimental results show that the current SOTA systems achieve 22.1%,
63.3%, and 83.5% for token-based exact-match, F1, and event-based HIT@1 scores,
which are all significantly below human performances (36.0%, 79.6%, 100%
respectively), highlighting our dataset as a challenging benchmark.
Understanding how events are semantically related to each other is the
essence of reading comprehension. Recent event-centric reading comprehension
datasets focus mostly on event arguments or temporal relations. While these
tasks partially evaluate machines' ability of narrative understanding,
human-like reading comprehension requires the capability to process event-based
information beyond arguments and temporal reasoning. For example, to understand
causality between events, we need to infer motivation or purpose; to establish
event hierarchy, we need to understand the composition of events. To facilitate
these tasks, we introduce ESTER, a comprehensive machine reading comprehension
(MRC) dataset for Event Semantic Relation Reasoning. The dataset leverages
natural language queries to reason about the five most common event semantic
relations, provides more than 6K questions and captures 10.1K event relation
pairs. Experimental results show that the current SOTA systems achieve 22.1%,
63.3%, and 83.5% for token-based exact-match, F1, and event-based HIT@1 scores,
which are all significantly below human performances (36.0%, 79.6%, 100%
respectively), highlighting our dataset as a challenging benchmark.
ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning
http://arxiv.org/abs/2104.08350v2