task_categories:conversational size_categories:10K<n<100K language:en license:apache-2.0 harmless region:us
---
license: apache-2.0
task_categories:
- conversational
language:
- en
tags:
- harmless
size_categories:
- 10K<n<100K
---

## Dataset Card for Anthropic_HH_Golden

This dataset is constructed to test the **ULMA** technique as mentioned in the paper *Unified Language Model Alignment with Demonstration and Point-wise Human Preference* (under review, and an arxiv link will be provided soon). They show that replacing the positive samples in a preference dataset by high-quality demonstration data (golden data) greatly improves the performance of various alignment methods (RLHF, DPO, ULMA). In particular, the ULMA method exploits the high-quality demonstration data in the preference dataset by treating the positive and negative samples differently, and boosting the performance by removing the KL regularizer for positive samples.


### Dataset Summary

This repository contains a new preference dataset extending the harmless dataset of Anthropic's Helpful and Harmless (HH) datasets. The origin positive response in HH is generated by a supervised fined-tuned model of Anthropic, where harmful and unhelpful responses are freqently encountered. In this dataset, the positive responses are replaced by re-rewritten responses generated by GPT4.

![Comparison with the origin HH dataset](https://cdn-uploads.huggingface.co/production/uploads/6516a787217abe5d7996dc7d/-q4koamraMoKYfluZ2o_y.png)

**Comparison with the origin HH dataset.** Left is the data sampled from the origin HH dataset, and right is the corresponding answer in our Anthropic_HH_Golden dataset. The highlighted parts are the differences. It is clear that after the rewritten, the "chosen" responses is more harmless, and the "rejected" response are left unchanged. 


### Usage
```
from datasets import load_dataset

# Load the harmless dataset with golden demonstration
dataset = load_dataset("Unified-Language-Model-Alignment/Anthropic_HH_Golden")

```
or download the data files directly with:
```
git clone https://huggingface.co/datasets/Unified-Language-Model-Alignment/Anthropic_HH_Golden
```