task_categories:text-generation multilinguality:monolingual size_categories:1K<n<10K source_datasets:extended|Anthropic/hh-rlhf language:en license:mit region:us
---
license: mit
language: en
multilinguality: monolingual
source_datasets: extended|Anthropic/hh-rlhf
task_categories:
- text-generation
size_categories:
- 1K<n<10K
---

dataset_info:
  features:
  - name: label
    dtype: string
  - name: input
    dtype: string
  - name: additional_info
    dtype: string


# Dataset Card for "AttaQ"

### Dataset Summary

Attack Questions (AttaQ) dataset is a set of adversarial examples in the form of questions designed to provoke harmful or inappropriate responses from LLM for red-teaming.
Red teaming goal is to evaluate LLM’s weaknesses that can lead to undesirable behaviors in real time. This is done by attacking LLM with adversarial attacks from this dataset and evaluating LLM’s response.

The datasets contains 1402 adversarial attacks for AI generative models, labeled to 7 categories (deception, discrimination, harmful information, substance abuse, sexual content, pii, violence).

AttaQ dataset is constructed from two parts:
First, we methodologically extracted attacks from  [Anthropic's hh-rlhf dataset](https://github.com/anthropics/hh-rlhf/tree/master/red-team-attempts) dataset and semi-automatically labeled to 7 categories.
Secondly, we generated actions per crime (from [Wikipedia Crimes Page](https://en.wikipedia.org/wiki/Crime)) and then use the actions to generate attack questions.



Warnings: 
1) The data contains offensive and upsetting content by nature therefore it may not be easy to read. Please read them in accordance with your own personal risk tolerance.
2) LLM's response to the AttaQ samples in many cases is harmful and/or violent. 
3) This dataset is a representative subset of all possible attacks. There are other attacks that can cause LLM to answer harmful or inappropriate responses.

Restrictions:
Red teaming community’s goal is making models less harmful. We restrict the usage of the dataset for making models less harmful.
	

### Data Fields
#### AttaQ
- `label`: corresponding label of adversarial question
- `input`: adversarial question
- `additional_info`: source of the adversarial question

### Citation Information
TBD

