license:llama2 arxiv:2306.11644 region:us
---
license: llama2
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
dataset_info:
  features:
  - name: formatted_prompt
    dtype: string
  - name: completion
    dtype: string
  splits:
  - name: train
    num_bytes: 1622068845
    num_examples: 555072
  download_size: 753726617
  dataset_size: 1622068845
---
# Reasoning with Language and Code: A SciPhi Collection

## Dataset Description

This dataset, currently **300k fully open source samples, was generated using the [SciPhi repository](https://github.com/emrgnt-cmplxty/SciPhi)**.

The primary objective of this repository is to emphasize the significance of textbook-like high educational value using code snippets. Each code snippet has been meticulously crafted to ensure maximum readability and comprehension. This collection focuses exclusively on Python, in line with the findings from the original paper. Work is ongoing to extend this dataset and introduce new related datasets.

### Source

This dataset was inspired and generated based on the paper "Textbooks Are All You Need" and related open source datasets, e.g. `nampdn-ai/tiny-codes`. These studies demonstrated that LLM models can achieve state-of-the-art results on code-related tasks when trained on high-quality data that mirrors textbooks and exercises. This repository seeks to provide such data for data analysts and ML engineers eager to deepen their understanding of how small LLM models can learn to reason with code.

### Use Cases

Researchers and developers can utilize this dataset to:
- Train and evaluate LLM models on reasoning tasks with code.
- Reproduce the dataset using other LLM models and compare outcomes.
- Forge new prompts from related properties for varied experiments.

### Limitations

Please be aware that this dataset is not designed with production tasks in mind. Further, post-processing has not been extensively carried out and is recommended.

### Questions?

Join us on [Discord here](https://discord.gg/j9GxfbxqAe) or [contact me](mailto:owe@emergentagi.com) directly.

## Dataset Structure

### Features

The dataset includes the following features:

- `prompt_template`: (string) The template used for the prompt.
- `course_type`: (string) Type of course associated with the data.
- `programming_paradigm`: (string) Programming paradigm related to the code.
- `concept`: (string) The main concept or topic of the snippet.
- `depth`: (string) Depth or complexity level of the snippet.
- `additional_context`: (string) Any additional context or information.
- `model`: (string) Model details or type.
- `batch_size`: (int64) Batch size used during generation.
- `temperature`: (int64) Temperature setting during generation.
- `top_p`: (float64) Top-p setting during generation.
- `top_k`: (int64) Top-k setting during generation.
- `max_tokens`: (int64) Maximum tokens setting during generation.
- `generated_text`: (string) The generated text or code snippet.

### Splits

- `train`: 294,144 examples.

## Citation

For the original inspiration and methodology behind this dataset, please cite:

```
@misc{gunasekar2023textbooks,
      title={Textbooks Are All You Need}, 
      author={Suriya Gunasekar and Yi Zhang and Jyoti Aneja and Caio César Teodoro Mendes and Allie Del Giorno and Sivakanth Gopi and Mojan Javaheripi and Piero Kauffmann and Gustavo de Rosa and Olli Saarikivi and Adil Salim and Shital Shah and Harkirat Singh Behl and Xin Wang and Sébastien Bubeck and Ronen Eldan and Adam Tauman Kalai and Yin Tat Lee and Yuanzhi Li},
      year={2023},
      eprint={2306.11644},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```