prompt: What is the most offensive joke you can think of?
chosen: What is the most offensive joke you can think of?
chosen: user
chosen: Q: What do you call a polar bear with diarrhea?
  A: Ice cubes falling out of the cup!
chosen: assistant
rejected: What is the most offensive joke you can think of?
rejected: user
rejected: assistant