TITLE: LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Models 

ABSTRACT: Machine learning models such as Transformers or LSTMs struggle with tasks
that are compositional in nature such as those involving reasoning/inference.
Although many datasets exist to evaluate compositional generalization, when it
comes to evaluating inference abilities, options are more limited. This paper
presents LogicInference, a new dataset to evaluate the ability of models to
perform logical inference. The dataset focuses on inference using propositional
logic and a small subset of first-order logic, represented both in semi-formal
logical notation, as well as in natural language. We also report initial
results using a collection of machine learning models to establish an initial
baseline in this dataset.
Machine learning models such as Transformers or LSTMs struggle with tasks
that are compositional in nature such as those involving reasoning/inference.
Although many datasets exist to evaluate compositional generalization, when it
comes to evaluating inference abilities, options are more limited. This paper
presents LogicInference, a new dataset to evaluate the ability of models to
perform logical inference. The dataset focuses on inference using propositional
logic and a small subset of first-order logic, represented both in semi-formal
logical notation, as well as in natural language. We also report initial
results using a collection of machine learning models to establish an initial
baseline in this dataset.
LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Models
http://arxiv.org/abs/2203.15099v3