**Safety and Alignment.** Despite the capacities, LLMs are faced with great safety challenges in practical use. As a fundamental issue of probabilistic modeling nature, LLMs exhibit a tendency to generate hallucinations [638], referring to texts that seem plausible but may be factually incorrect [46]. What is worse, LLMs might be elicited by intentional instructions to produce harmful, biased, or toxic texts for malicious systems, leading to the potential risks of misuse [66, 55]. To have a detailed discussion of the safety issues of LLMs (_e.g._, privacy, overreliance, disinformation, and influence operations), the readers can refer to the GPT-3/4 technical reports [55, 46]. As the major technical approach to averting these issues, alignment methods (_e.g._, RLHF) [116, 66] have been widely used by leveraging human feedback for developing well-aligned LLMs. However, RLHF heavily relies on high-quality human feedback data from professional labelers, which is costly and time-consuming to recruit qualified human annotators. Therefore, it is necessary to improve the RLHF framework for reducing the efforts of human labelers and seek a more efficient annotation approach with guaranteed data quality, _e.g._, LLMs can be employed to assist the labeling work. Furthermore, it is also suggested to develop simplified optimization algorithms for alignment [386, 389], to reduce the training difficulty and unstability of RLHF. As another practical approach, red teaming [369, 132] has been adopted for improving the model safety of LLMs, which utilizes the collected adversarial prompts to refine the LLMs (_i.e._, avoiding the attacks from red teaming). In addition, privacy concerns are also important to consider when fine-tuning LLMs with domain-specific data, and thus federated based learning [946] can be useful in privacy-restricted scenarios.