arxiv:2305.18290 region:us
---
# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/datasetcard.md?plain=1
# Doc / guide: https://huggingface.co/docs/hub/datasets-cards
{}
---

# Dataset Card for LeetCode Preference

## Dataset Description

- **Homepage:** 
- **Repository:** 
- **Paper:** 
- **Leaderboard:** 
- **Point of Contact:** 

### Dataset Summary
This dataset facilitates experiments utilizing Direct Preference Optimization (DPO) as outlined in the paper titled [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://arxiv.org/abs/2305.18290). This repository provides code pairings crafted by CodeLLaMA-7b. For every LeetCode question posed, CodeLLaMA-7b produces two unique solutions. These are subsequently evaluated and ranked by human experts based on their accuracy, efficiency, and readability.

### Usage 
```
from datasets import load_dataset

dataset = load_dataset("minfeng-ai/leetcode_preference")
```


### Data Fields

Each row of the dataset contains the following fields:
* id: A distinct identifier assigned to each LeetCode question.
* title: The official title of the LeetCode question.
* description: An in-depth prompt offering detailed insights into the respective question.
* difficulty: Categorized into three tiers, indicating the complexity of the question - Easy, Medium, and Hard.
* version1: The initial AI-generated code snippet pertaining to the question.
* version2: A secondary AI-generated code snippet related to the same question.
* preference: A human-assessed label indicating the preferred code snippet between Version1 and Version2.