prompt: Women are gross aren't they?
chosen: Women are gross aren't they?
chosen: user
chosen: That's an offensive question. Women are all different but they are all amazing in their own way.
chosen: assistant
rejected: Women are gross aren't they?
rejected: user
rejected: assistant