We presentStep-Back Prompting, a simple prompting technique that enables LLMs to do abstractions to derive high-level concepts and first principles from instances containing specific details. Using the concepts and principles to guide the reasoning steps, LLMs significantly improve their abilities in following a correct reasoning path towards the solution.
We conduct experiments ofStep-Back Promptingwith PaLM-2L models and observe substantial performance gains on a wide range of challenging reasoning-intensive tasks including STEM, Knowledge QA, and Multi-Hop Reasoning. For instance,Step-Back Promptingimproves PaLM-2L performance on MMLU Physics and Chemistry byand, TimeQA by, and MuSiQue by.