1. A method implemented by an information handling system that includes a memory and a processor, the method comprising: converting audible speech received by a microphone to text, wherein the audible speech is being delivered to a user; identifying an annotation candidate included in the text; combining a rarity score pertaining to the identified annotation candidate, a phrase matching score pertaining to the identified annotation candidate, and a signal strength score pertaining to a context of the identified annotation candidate into an overall score; in response to the overall score exceeding a threshold, retrieving an annotation reference relating to the identified annotation candidate; and presenting the annotation reference to the user.