Hyper-Multilingual LLMsFrom Figure 0(b) we observe that BLOOMZ also tends to default to English as its "thinking" language. Therefore, it can support multiple languages mainly because it has the ability to understand and generate these languages while using English as the language of thinking. Additionally, Figure 4 illustrates the degree of neuron overlap among languages within both the self-attention and feed-forward structures of BLOOMZ. In contrast to the findings shown in Figure 3, there is a marked reduction in overlap, indicating that individual languages maintain a higher degree of independence and do not extensively share neurons with one another.