<commit_before># LogicInference Dataset

This repository contains the Python code used to generate the `LogicInference`
dataset. `LogicInference` is a dataset designed to evaluate the ability of
models to perform logical inference. The dataset focuses on inference using
propositional logic and a small subset of first-order logic, represented both in
semi-formal logical notation, and in natural language. `LogicInference` has two
main long-term goals: (1) to evaluate the ability of models to perform logical
inference, and the degree to which inference chains are real or hallucinated,
and (2) to assess whether learning logical inference abilities in the abstract
(e.g., getting better in this dataset) would then transfer to other real-world
tasks.

For a detailed description of the dataset, please check the following paper:
https://arxiv.org/abs/2203.15099]

Please cite as:

```
@misc{ontanon2022logicinference,<commit_msg>Fixed a typo on the arXiv URL<commit_after># LogicInference Dataset

This repository contains the Python code used to generate the `LogicInference`
dataset. `LogicInference` is a dataset designed to evaluate the ability of
models to perform logical inference. The dataset focuses on inference using
propositional logic and a small subset of first-order logic, represented both in
semi-formal logical notation, and in natural language. `LogicInference` has two
main long-term goals: (1) to evaluate the ability of models to perform logical
inference, and the degree to which inference chains are real or hallucinated,
and (2) to assess whether learning logical inference abilities in the abstract
(e.g., getting better in this dataset) would then transfer to other real-world
tasks.

For a detailed description of the dataset, please check the following paper:
https://arxiv.org/abs/2203.15099

Please cite as:

```
@misc{ontanon2022logicinference,
{'fraction_non_alphanumeric': 0.05162320383182544, 'fraction_numerical': 0.015965939329430547, 'mean_word_length': 5.32996632996633, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2343160', 'n_tokens_mistral': 505, 'n_tokens_neox': 462, 'n_words': 255}