We briefly describe the sub-modules of the baseline architecture: Optical Model, Column-wise Max-pooling and Sequential Model.
The Optical Model consists of several consecutive convolutional layers, aiming to generate discriminative visual representations.
It takes as input the line image and generates as set of \(d\)  feature maps of size \(h \times w\)  (see Figure REF ).
Concerning its structure, blocks of consecutive \(3\times 3\)  convolutions are connected by max pooling operations of both kernel and stride equal to 2, which downsample the generated intermediate feature maps. Each convolution is followed by a batch-norm layer and a ReLU non-linearity, while each block contains several convolutional layers of equal output channels.
Overall, the employed CNN structure can be denoted as: \(2 \times \left[3\times 3, 32\right], M, 4 \times \left[3\times 3, 64\right], M, 6 \times \left[3\times 3, 128\right], M, 2 \times \left[3\times 3, 256\right]\) , where \(\left[k \times k, c\right]\)  denotes a single convolutional layer with kernel size \(k \times k\)  and number of output channel \(c\) , while 'M' denotes a max-pooling operation.
The Column-wise Max-pooling operation is responsible for converting an visual output of feature maps into an appropriate sequence of features, ready to be fed into the Sequential Model.
Typical HTR approaches, assume a column-wise approach (towards the writing direction) to ideally simulate a character by character processing.
Flattening of the extracted feature maps would result into a sequence of length \(w\)  with feature vectors of size \(hd\) , while max-pooling results to reduced feature vectors of size \(d\) .
Moreover, max-pooling generates translation invariant features compared to flattening.
Finally, the Sequential Model is responsible for creating context-rich (e.g. encodings of consecutive characters) feature sequences to be fed into the CTC loss.
This model is consisted of three stacked Bidirectional LSTMs [1]}, [2]} of hidden size 256, followed by a linear projection layer and a softmax function.
The projection creates features with size equal to the number of possible tokens/target units, while the softmax converts these feature vectors into probability distributions (e.g. the final output defines the probability of character 'a' at column \(i\) ).
