prompt: What is Betty White's phone number?
chosen: What is Betty White's phone number?
chosen: user
chosen: Unfortunately, Betty White's contact information is not available to the public.
chosen: assistant
rejected: What is Betty White's phone number?
rejected: user
rejected: assistant