**Overview.** The basic idea of non-RL alignment approaches is to directly fine-tune LLMs with _supervised learning_ on high-quality _alignment dataset_. It basically assumes that response feedback or golden rules to avert unsafe behaviors have been injected or included in the specially curated alignment dataset, so that LLMs can directly learn aligned behaviors from these demonstration data via suitable fine-tuning strategies. Thus, to implement this approach, two key issues are the construction of alignment dataset and the design of fine-tuning loss. For the first issue, the alignment dataset can be automatically constructed by an aligned LLMs according to human-written safety principles [347] or refining existing examples using edits operations [383]. In addition, we can also reuse existing reward models to select high-rated responses from existing human feedback data [373]. For the second issue, non-RL alignment approaches mainly fine-tune LLMs in a supervised learning way (the same as the original instruction tuning loss) on a high-quality alignment dataset, meanwhile auxiliary learning objectives can be used to enhance the alignment performance, _e.g._, ranking responses or contrasting instruction-response pairs.