In this work, we aim to enhance the alignment of non-English LLMs with human preferences. Inspired by the concept of task vectors (Ilharco et al., 2023), we hypothesize that given a consistent base model, pre-existing knowledge and acquired behaviors can be synergized through a straightforward vector addition in the parameter space. To achieve this, we propose an approach to restructure the conventional training paradigm for non-English LLMs from CP \(\rightarrow\) SFT \(\rightarrow\) RLHF to CP \(+\) chat vector. The chat vector is derived by subtracting LLaMA-2's pre-trained weights from those of its chat-enhanced counterpart, LLaMA-2-chat. By introducing this chat vector to a LLaMA-2-based model that's continually pre-trained on non-English content, the evolved model responds in the target language, both in providing answers and declining inappropriate requests, and it aligns more deeply with human preferences. The main process of our method is illustrated in Figure 1.