task_categories:text-generation size_categories:10K<n<100K language:en license:cc-by-nc-4.0 region:us
---
license: cc-by-nc-4.0
viewer: true
task_categories:
- text-generation
language:
- en
pretty_name: MALLS NL-FOL Pairs 34K
size_categories:
- 10K<n<100K
---

# MALLS NL-FOL Pairs 34K

## Dataset details

MALLS (large language **M**odel gener**A**ted natural-**L**anguage-to-first-order-**L**ogic pair**S**) 
consists of 34K pairs of real-world natural language (NL) statements and the corresponding first-order logic (FOL) rules annotations.
All pairs are generated by prompting GPT-4 and processed to ensure the validity of the FOL rules.

Note that we did not conduct a rigorous alignment check on the pairs, meaning the FOL rule may not accurately reflect the meaning of the NL statement.
That said, we recommend treating the dataset as "silver" labels and using it for training, and using another dataset with "gold" labels for evaluation.

# Dataset Structure

The file `MALLS-v0.json` consists of the 34K pairs of the MALLS dataset; we also provide `folio_parsed.json` which consists of 2K pairs collected
and processed from the FOLIO datset. Each entry in the file is a dictionary object of the following format
```
{
  'NL': <the NL statment>,
  'FOL': <the FOL rule>
}
```

**License:**
Attribution-NonCommercial 4.0 International.
Since the data are collected from GPT-4, it also abides by the policy of OpenAI: https://openai.com/policies/terms-of-use

## Using the Dataset

We use MALLS to finetune a LLaMA-7B model for NL-FOL translation, namely LogicLLaMA, which achieves GPT-4 level performance.

**Project Page**
https://github.com/gblackout/LogicLLaMA


## Intended use
**Primary intended uses:**
MALLS is intended to be used for research.

## Citation
```
@article{yang2023harnessing,
      title={Harnessing the Power of Large Language Models for Natural Language to First-Order Logic Translation}, 
      author={Yuan Yang and Siheng Xiong and Ali Payani and Ehsan Shareghi and Faramarz Fekri},
      journal={arXiv preprint arXiv:2305.15541},
      year={2023}
}
```