First, we consider the case of unigram-level greedy decoding.
State-of-the-art results along with the performance of our proposed models appear in Table REF .
Notably, our single-task architecture outperforms both related works from the literature.
specifically, Pham et al. [1]} utilized 2D-BiLSTMs, while Puigcerver et al. [2]} used a similar architecture to the proposed single-task baseline, whilst having twice as many parameters compared to ours.
Table REF  also contains the evaluation of the different multi-task variants at different granularities, using the unigram level greedy decoding scheme.
As reported, we utilize only the unigram posteriors so as to assess the impact of the multi-task reasoning on the internal representations of the trained model.
From the results, it is evident that the use of this extra n-gram information, only on the training step, can significantly boost the model's ability to generalize and consequently the system's performance.
Indeed, Learning in the terms of training multiple CTC decompositions of different granularity, from fine-to-coarse target units, boosts the model's ability to generalize and thus we report improvement in the WER and CER metrics.
Moreover, as the language information increases by using higher n-gram scales we obtain better results.
We should stress that only the bigram information results to a considerable boost, while extra n-gram scale provide slight, yet consistent, performance increase.
Note that all reported variants have the same complexity, since only the unigram branch is used for evaluation.
Furthermore, no important difference among the HMT and BMT alternatives is observed, even though conceptually the former describes better the contextual nature of the n-grams.
This can be addressed to the fact that HMT has a more complex structure which may affect the back-propagation process and hinder optimization/training, since each n-gram level straightforwardly affects its predecessor.
Next, we compare our unigram-level greedy decodings with the ones from previous papers. We refer to a 1D-LSTM approach [2]} as the LSTMs utilized in our architecture and one architecture [1]} that utilized 2D-LSTM.
Summarizing, our multi-task architectures significantly outperform the majority existing approaches at the greedy decoding setting, while being on par with a very recent and computational demanding sequence to sequence approach [5]}.
Specifically, our best performing variant, i.e. BMT with all n-gram scales, which will be the default network for further evaluation, has an improvement of absolute 2.52% in the WER and 1.02% in the CER compared to the very similar network of  [2]}, by just utilizing the unigram-level posteriors, without any extra computational overhead.
<TABLE>