1. A method for updating a first speech model in a speech recognition system, comprising: receiving from a user device of a user, the user device comprising a personal speech model trained for the user through previous speech recognition operations, one or more personal speech model components of the personal speech model trained for the user through previous speech recognition operations, the one or more personal speech model components describing personal speech characteristics of the user; updating the first speech model using at least some of the one or more personal speech model components, by modifying at least one speech model component of the first speech model and/or adding at least one speech model component to the first speech model; and performing speech recognition on user speech using the first speech model updated with the at least some of the one or more personal speech model components.