A context sensitive chunk BPTT approach to training deep LSTM/BLSTM recurrent neural networks for offline handwriting recognition
We propose a context sensitive chunk based back propagation through time (BPTT) approach to training deep (bidirectional) long short term memory (B)LSTM) recurrent neural networks (RNN) that splits each training sequence into chunks with appended contextual observations for character modeling of offline handwriting recognition. Using short context sensitive chunks in both training and recognition brings following benefits: (1) the learned (B)LSTM will model mainly local character image dependency and the effect of long range language model information reflected in training data is reduced; (2) mini batch based training on GPU can be made more efficient; (3) low latency BLSTM based handwriting recognition is made possible by incurring only a delay of a short chunk rather than a whole sentence. Our approach is evaluated on IAM offline handwriting recognition benchmark task and performs better than the previous state of the art BPTT based approaches.
2015 13th International Conference on Document Analysis and Recognition (ICDAR)