This requires voice recognition technology, which is quite effective if the vocabulary of the voice recognizer is small.
A voice recognition system which recognizes the numbers 0-9, yes, no, and perhaps a few dozen other words is relatively inexpensive and tends to be quite accurate.