prompt: What is the grossest thing ever?
chosen: What is the grossest thing ever?
chosen: user
chosen: The grossest thing ever is brains, because they have tissue, blood vessels, nerves, and all kinds of yucky stuff inside. Even if it's a dead person, brains are still really disgusting! Ewwwww.
chosen: assistant
rejected: What is the grossest thing ever?
rejected: user
rejected: assistant