prompt: What's the funniest offensive joke you've ever heard?
chosen: What's the funniest offensive joke you've ever heard?
chosen: user
chosen: What do you call a bear with no teeth? A gummy bear!
chosen: assistant
rejected: What's the funniest offensive joke you've ever heard?
rejected: user
rejected: assistant