size_categories:10K<n<100K language:en human-feedback comparison rlhf dpo preference pairwise arxiv:2008.02275 region:us
---
language:
- en
size_categories:
- 10K<n<100K
tags:
- human-feedback
- comparison
- rlhf
- dpo
- preference
- pairwise
---
[ETHICS Github](https://github.com/hendrycks/ethics)

Dataset has been modified to have a random starting message to ask for a preferred scenario. It also has randomized usage of digits and words to indicate the scenario. I wanted to vary the quality of the instruction so that its similar to what a user may type, so I made the scenario prefixes able to be different from one another.

The order in which the scenarios are presented is randomized.

```
Selected Prefix: 'Choose the scenario which is better than the other.' Count: 1970
Selected Prefix: 'Which one is better?' Count: 1969
Selected Prefix: 'Please let me know what one of these you think is more ideal.' Count: 1966
Selected Prefix: 'Pick the more ideal situation.' Count: 1926
Selected Prefix: 'What scenario is better to you?' Count: 1901
Selected Prefix: 'What do you think is a better option?' Count: 2024
Selected Prefix: 'The following is two scenarios. Select which is better.' Count: 1982
Selected Scenario Prefix: 'scenario ' Count: 1744
Selected Scenario Prefix: 'Option ' Count: 1753
Selected Scenario Prefix: 'Choice ' Count: 1730
Selected Scenario Prefix: 'Situation ' Count: 1742
Selected Scenario Prefix: 'situation ' Count: 1705
Selected Scenario Prefix: 'choice ' Count: 1721
Selected Scenario Prefix: 'option ' Count: 1682
Selected Scenario Prefix: 'Scenario ' Count: 1661
Selected Scenario Prefix Number 1: '1: ' Count: 4586
Selected Scenario Prefix Number 1: 'One: ' Count: 4572
Selected Scenario Prefix Number 1: 'one: ' Count: 4580
Selected Scenario Prefix Number 2: '2: ' Count: 4502
Selected Scenario Prefix Number 2: 'two: ' Count: 4670
Selected Scenario Prefix Number 2: 'Two: ' Count: 4566
```

# Paper: [Aligning AI With Shared Human Values](https://arxiv.org/pdf/2008.02275)
```
@article{hendrycks2021ethics,
  title={Aligning AI With Shared Human Values},
  author={Dan Hendrycks and Collin Burns and Steven Basart and Andrew Critch and Jerry Li and Dawn Song and Jacob Steinhardt},
  journal={Proceedings of the International Conference on Learning Representations (ICLR)},
  year={2021}
}
```