In fact, the number of parameters is only a small multiple of the compact model (the multiplication factor accounts for the language embedding size)<cite> (Johnson et al., 2017)</cite> , but the language embeddings can directly impact the model parameters instead of the weak influence that language tags have.