1. A method comprising: outputting, by a computing device and for display at an output component operatively coupled to the computing device, a graphical user interface that includes a text edit region and a graphical keyboard; receiving, by the computing device, an indication of a first input detected at a first location of a presence-sensitive input component operatively coupled to the computing device, wherein the first location of the presence-sensitive input component corresponds to a region of the output component at which the graphical keyboard is displayed; determining, by the computing device, based at least in part on the indication of the first input, a new character string that is not included in a language model of the computing device; adding, by the computing device, to the language model, the new character string and a corresponding likelihood value associated with the new character string; receiving, by the computing device, an indication of a second input detected at a second location of the presence-sensitive input component, wherein the second location of the presence-sensitive input component corresponds to the region of the output component at which the graphical keyboard is displayed; predicting, based on the indication of the second input and the likelihood value associated with the new character string, the new character string and an input context associated with the second input; responsive to predicting the new character string, outputting, by the computing device, for display within the text edit region, the new character string; receiving, by the computing device, an indication of a third input detected at a third location of the presence-sensitive input component, wherein the third location of the presence-sensitive input component corresponds to the region of the output component at which the graphical keyboard is displayed; and responsive to determining that the third input deletes the new character string from the text input region after being output for display: removing, by the computing device, from the text input region, the new character string; and decreasing, by the computing device, at a rate or magnitude that is adapted to the input context, the likelihood value associated with the new character string without removing the new character string from the language model.