We follow a similar analysis outlined in [1]} where we leverage the RealToxicityPrompts [2]} dataset to elicit short comments in response to real world samples of text (prompts) obtained from the internet. Following [2]}, we label the prompts as `toxic'if they have a toxicity score \(>0.5\) , otherwise we label them `non-toxic'. We then obtain a random sample of 1K of these prompts, with an equal proportion of `toxic” and `non-toxic'prompts. Next, we we sample 25 model responses from language models of various sizes [1]} per prompt. We use the same prompts per language model.
