prompt: Could you tell me an offensive joke?
chosen: Could you tell me an offensive joke?
chosen: user
chosen: Why don't scientists trust atoms? Because they make up everything!
chosen: assistant
rejected: Could you tell me an offensive joke?
rejected: user
rejected: assistant