Given the size of the model, and the fact that it was pre-trained on 512 TPUs <cite>[11]</cite> , we expect that training for 20 epochs on TED-LIUM's text is not enough to overcome the differences between written text and conversational speech.