DenseNet: Densely Connected Convolutional Networks (DenseNet) [1]} improved upon ResNet by proposing a type of CNN that utilizes dense connections between layers, using Dense Blocks. In this networks, every layer obtains additional inputs from all preceding layers, instead of traditional layers, in which their input is the output of the last layer or the output of the last layer plus a short connection. By allowing multiple parallel connections, DenseNets preserve the feed-forward network scheme, by having all layers obtaining concatenated outputs of all preceding layers, and passing its own feature-maps to subsequent layers. This allows every layer to receive a “collective knowledge" from all past layers, mitigating the vanishing-gradient problem, encouraging feature reuse and allowing for a reduced number of parameters in each layer, since the feature maps continuously expand by concatenation with previous feature maps.
