The dimension variation at different stages of LeNet-5 is shown in Fig.
REF , where a red upward arrow denotes dimension
expansion and a blue downward arrow denotes dimension reduction.
Dimensions increase with convolution operations in the first two layers,
and dimension decreases with the (2x2)-to-(1x1) pooling operations. The
increase in the dimension of an intermediate layer is determined by the
network architecture. By adding more neurons and reducing the stride
number, the intermediate layer will have more dimensions. More embedding
variables allow the generation of more expressive embeddings. The
design of network architecture highly depends on specific applications
(or datasets). The search of an optimal network architecture
[1]} and the associated loss function for new
problems is costly.
