If we focus on the encoder neural networks, previous studies benefit from fusing multiple features in different layers[1]}[2]}[3]}.
On CNNs in particular, it is known that early layers, local features, contain relatively "general" filters, whereas deeper layers, global features, are specific to the training dataset[4]}. These suggest that multi-aspect features available in layers potentially serve various needs.
