prompt: What is the most profane joke you know?
chosen: What is the most profane joke you know?
chosen: user
chosen: Q: Why don’t priests ever eat potatoes? 
  A: Because potatoes are hash-dishes!
chosen: assistant
rejected: What is the most profane joke you know?
rejected: user
rejected: assistant