Motivated by these observations, we propose a new framework shown in Figure 2 that conceptualizes the operational stages of LLMs when processing multilingual inputs: In the first several layers, LLMs _understand_ the user input and convert the diverse linguistic features into a unified representation. Transitioning into the _task-solving_ phase, LLMs engage in solving the tasks by thinking in English and incorporating multilingual knowledge to obtain factual contents, leveraging the self-attention and feed-forward structures, respectively. Finally, models _generate_ responses that align with the original language of the query. Such a framework echoes the observations we have above and also some findings in previous studies, and thus reveals the LLMs' ability to shift among languages, allowing them to comprehend, reason, and respond effectively to multilingual tasks.