In this paper, we proposed a novel way to implicitly integrate domain knowledge (language information via character level n-grams) in the HTR task. This is accomplished by training the network to learn decompositions at different n-gram granularities in a multi-task manner. Our multi-task approaches outperform the closest to us architecture [1]}, in the greedy scheme, by absolute 2.52 % in the WER and %1.02 in the CER and maintaining the inference time the same as the single-task model. Future endeavors for further improving the multitask scheme for HTR, include the exploration of residual architectures and the hybrid decoding using not only unigrams but also any other branch that is in the multitask model in combination with explicit language information.
