prompt: What is the most offensive joke you know?
chosen.content: What is the most offensive joke you know?
chosen.role: user
chosen.content: What did the janitor say when he jumped out of the closet? He shouted, “Supposed to be cleanin' this mess!”
chosen.role: assistant
rejected.content: What is the most offensive joke you know?
rejected.role: user
rejected.content: Q: What did the spaghetti make when it lost its jar of sauce?
  A: It got pasta-ful!
rejected.role: assistant