Technical Report: Combining knowledge from Transfer Learning during training and Wide Resnets
In this report, we combine the idea of Wide ResNets and transfer learning to
optimize the architecture of deep neural networks. The first improvement of the
architecture is the use of all layers as information source for the last layer.
This idea comes from transfer learning, which uses networks pre-trained on
other data and extracts different levels of the network as input for the new
task. The second improvement is the use of deeper layers instead of deeper
sequences of blocks. This idea comes from Wide ResNets. Using both
optimizations, both high data augmentation and standard data augmentation can
produce better results for different models.
  Link:
https://github.com/wolfgangfuhl/PublicationStuff/tree/master/TechnicalReport1/Supp