In this study, we focus on how to improve the CTC with word output units. There are two challenges to the word-based CTC. The first one is the OOV issue. In <cite>[17]</cite> [23] [24] , only frequent words in the training set are used as the targets and the remaining words are just tagged as the OOV.