Feedback module vs Language Model: We could have substituted our iterative approach towards refining text prediction with a Language model(LM). For fairness, we use a state-of-the-art RNN-LM [1]} trained from text corpus (librispeech) at character level [2]} that aims to predict the next likely character. This could be fused with the text recognition decoder using two state-of-the-art methods introduced in [3]} via Shallow Fusion (weighted sum of predicted scores) and Deep Fusion (fusing their hidden states). Our method performs better in both CS and DS setups in comparison to these LM integrations (Table REF ). The limited performance of off-the-shelf LM [4]} can be attributed to: (i) LM is mostly used in speech recognition tasks, where data is present at sentence-level which provides enough context. For distract word recognition (our focus) however, LM cannot harness such extent of context information. (ii) The LM corpus is significantly different from that used for training word-image recognition system. This leads to a biased incorrectness [1]}.
(iii) LM being an independent post processing step, not only ignores rich visual features from the input image, but is also unaware of the error distribution of the model. On the contrary, our model revisits the rich visual features iteratively after every prediction, considering the error distribution while training. Furthermore to align with the evaluation standards for unconstrained word recognition we cite all results in our work using greedy decoding only – no LM based post-processing.
<TABLE>