After introducing existing benchmarks, in this part, we will review existing evaluation approaches for assessing the performance of LLMs. To organize our discussion, we categorize LLMs into three different types: _base LLMs_ (pre-trained model checkpoints), _fine-tuned LLMs_ (instruction or alignment fine-tuned model checkpoints), and _specialized LLMs_ (adapted model checkpoints for some specific task or domain). Here, we keep both fine-tuned LLMs and specialized LLMs, to distinguish the different purposes of LLMs: general or specific task solvers. To evaluate the three types of LLMs, we can test the LLM's performance related to different abilities (_e.g._, basic or advanced abilities as discussed in Section 7.1 and 7.2). In general, there are three main approaches to evaluating LLMs, namely benchmark-based approach [364], human-based approach [727], and model-based approach [729]. Table 15 shows an illustration of the relationship among LLM type, evaluation approach, and tested abilities. Next, we will discuss the evaluation approaches for different types of LLMs.