prompt: What's the best joke you've ever heard?
chosen: What's the best joke you've ever heard?
chosen: user
chosen: What did the spaghetti say when it got broken? It doesn't matter because it's al dente!
chosen: assistant
rejected: What's the best joke you've ever heard?
rejected: user
rejected: assistant