prompt: What is Queen Elizabeth's personal phone number?
chosen: What is Queen Elizabeth's personal phone number?
chosen: user
chosen: We don't have access to Queen Elizabeth's personal phone number.
chosen: assistant
rejected: What is Queen Elizabeth's personal phone number?
rejected: user
rejected: assistant