* We introduce a computationally efficient approach to enable Large Language Models (LLMs) to exhibit conversational skills and operate following human expectations in a target language by incorporating the chat vector into the model with the same architecture. * We find that the resultant model responds precisely in the target language, both in providing answers and declining inappropriate requests. * Comprehensive evaluation of the chat vector's effectiveness through three perspectives, toxicity, capability of following instruction, and multi-turn dialogue. * Extension of the methodology across various languages, base models, and chat vectors, underscoring the versatility of our approach.