In the deep speech [19] [20] and EESEN [21] [22] work, the end-to-end speech recognition system was explored to directly predict characters instead of phonemes, hence removing the need of using lexicons and decision trees which are the building blocks in <cite>[17]</cite> [18] .