task_categories:question-answering size_categories:n<1K language:en license:mit question answering math science visual question answering arxiv:2305.12524 region:us
---
license: mit
task_categories:
- question-answering
language:
- en
tags:
- question answering
- math
- science
- visual question answering
pretty_name: ThoeremQA
size_categories:
- n<1K
---

## Introduction
We propose the first question-answering dataset driven by STEM theorems. We annotated 800 QA pairs covering 350+ theorems spanning across Math, EE&CS, Physics and Finance. The dataset is collected by human experts with very high quality. We provide the dataset as a new benchmark to test the limit of large language models to apply theorems to solve challenging university-level questions. We provide a pipeline in the following to prompt LLMs and evaluate their outputs with WolframAlpha.

## How to use TheoremQA
```
from datasets import load_dataset

dataset = load_dataset("wenhu/TheoremQA")

for d in dataset['test']:
  print(d)
```
To use the images, try to download images from images.zip in https://huggingface.co/datasets/wenhu/TheoremQA/blob/main/images.zip. The image is under the `Picture' field.

## Arxiv Paper:
https://arxiv.org/abs/2305.12524

## Code
https://github.com/wenhuchen/TheoremQA/tree/main