DL is characterized by two design choices: the network architecture and
the loss function. Once these two choices are specified, model
parameters can be automatically determined via an end-to-end
optimization algorithm called backpropagation. When the number of
training samples is less than the number of model parameters, it is
common to adopt pre-trained networks to build larger networks for better
performance; e.g., ResNet [1]} or DenseNet
[2]} pretrained by ImageNet [3]}.
Another emerging trend is the adoption of the transformer architecture
[4]}, [5]}, [6]}, [7]}, [8]}.
