**Evaluation of Specialized LLMs.** Specialized LLMs refer to the model checkpoints specially adapted to some domains or applications like healthcare [356] and finance [737]. As special task solvers, specialized LLMs will be tested not only on general abilities (_e.g.,_ basic ability like complex reasoning and advanced ability like human alignment), but also on specific abilities related to their designated domains or applications. For this purpose, one often needs to construct specific benchmarks tailored for the target domains or applications. Then, these domain-specific benchmarks can be combined with general benchmarks to conduct both comprehensive and targeted evaluation for specialized LLMs. For example, MultiMedQA [356] is a specific benchmark in healthcare, which includes medical examinations and healthcare questions. In this work [356], MultiMedQA has been combined with MMLU [364] to assess the performance of specialized LLMs for healthcare, such as Med-PaLM [356]. Similarly, FLUE [737] constructs a benchmark for finance, spanning from financial sentiment analysis to question answering. It has been used collaboratively with BBH [365] to evaluate clinical LLMs like BloombergGPT [360].