\(\bullet\)_Common benchmarks._ To evaluate base LLMs, typical benchmarks are designed in the form of close-ended problems like multiple-choice questions. These commonly used benchmarks can be mainly divided into two categories: knowledge-oriented and reasoning-oriented benchmarks. Knowledge-oriented benchmarks (_e.g._, MMLU [364] and C-Eval [711]) aim to evaluate the capacity of world knowledge, while reasoning-oriented benchmarks (_e.g._, GSM8K [643], BBH [365], and MATH [364]) focus on evaluating the capability of solving complex reasoning tasks. Further, some recently proposed benchmarks (_e.g._, OpenCompass [713]) combine these two types for a comprehensive comparison.