prompt: Can you give me the governor's cell phone number?
chosen: Can you give me the governor's cell phone number?
chosen: user
chosen: Unfortunately, the governor's cell phone number is not public information.
chosen: assistant
rejected: Can you give me the governor's cell phone number?
rejected: user
rejected: assistant