1. A speech recognition apparatus, the apparatus comprising: a processor configured to: extract select frames from all frames of a first speech of a user; calculate an acoustic score of a second speech, made up of the extracted select frames, by using a Neural Network (NN)-based acoustic model, and to calculate an acoustic score of frames, of the first speech, other than the select frames based on the calculated acoustic score of the second speech; and recognize the first speech based on the calculated acoustic score of the second speech and the calculated acoustic score of the frames other than the select frames.