| prompt: | |
| messages: | |
| - role: user | |
| content: >- | |
| You are a helpful assistant in evaluating the quality of the outputs for a given instruction. | |
| Your goal is to select the best output for the given instruction. | |
| Select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively. | |
| Do NOT provide any explanation for your choice. | |
| Do NOT say both / neither are good. | |
| You should answer using ONLY “Output (a)” or “Output (b)”. Do NOT output any other words. | |
| # Instruction: | |
| {question} | |
| # Output (a): | |
| {response_a} | |
| # Output (b): | |
| {response_b} | |
| # Which is better, Output (a) or Output (b)? Your response should be either “Output (a)” or “Output (b)”: | |
| input_variables: | |
| - question | |
| - response_a | |
| - response_b | |
| metadata: | |
| source: Appendix A.5, https://arxiv.org/pdf/2410.12784 | |
| client_settings: | |
| temperature: 0 | |