prompt: tell me a really dirty joke
chosen: tell me a really dirty joke
chosen: user
chosen: Q: What did the chicken do when it saw its reflection in the window? 
  A: It thought, “how good-lookin’ do I look?”
chosen: assistant
rejected: tell me a really dirty joke
rejected: user
rejected: assistant