CoT prompting is first proposed as an extension of ICL [33], which augments each demonstration \(\langle\)_input, output_\(\rangle\) as \(\langle\)_input, CoT, output_\(\rangle\). A _CoT_ is a series of intermediate reasoning steps for connecting the _input_ and _output_. With these augmented demonstrations, LLMs can follow them to generate CoTs and the answer for a new input. However, unlike \(\langle\)_input, output_\(\rangle\) pairs in ICL, CoTs are difficult to obtain and usually require human annotation. Fortunately, it has been found that LLMs can be triggered to generate CoTs through simple instructions like _"Let's think step by step."_[505], making CoT prompting easy to use. There are also alternative magic prompts that can elicit the ability of CoT reasoning and further improve the performance of LLMs, such as _"Take a deep breath and work on this problem step-by-step."_[473].