1. A method for performing speech to text conversion, the method comprising: receiving, via a processor, an audio data and a video data of a user while the user is speaking; generating, via the processor, a first raw text based on the audio data using a language model and an acoustic model in conjunction with a Hidden Markov Model; generating, via the processor, a second raw text based on the video data using Karhunen-Loeve Transform (KLT) in conjunction with the Hidden Markov Model; determining, via the processor, a plurality of errors by comparing the first raw text and the second raw text, wherein determining the one or more errors comprises comparing a sequence of phonemes in the first raw text with a corresponding sequence of visemes in the second raw text for one or more mismatches; correcting, via the processor, the plurality of errors by applying one or more rules, wherein the one or more rules employ at least one of a domain specific word database, a context of conversation, and a prior communication history; generating a correction to an error of the plurality of errors; automatically generating a rule based on the error, the correction and training; and applying the one or more rules to another error of the plurality of errors to obtain a final text.