Skip connections and Network Science: Currently, both networks obtained by manual design and NAS have shown that long-range links (i.e., skip connections) are crucial for getting higher accuracy [1]}, [2]}, [3]}, [4]}. Overall, there are two commonly used skip connections in neural networks. First, we have the DenseNet-type skip connections (DTSC), which concatenate previous layers' outputs as the input for the next layer [2]}. To study the topological properties and enlarge the search space, we do not use the original DesneNets [2]}, which contains all-to-all connections. Instead, we consider a generalized version where we vary the number of skip connections by randomly selecting only some channels for concatenation, as shown in Fig. REF (a). The other type of skip connections is the addition-type skip connections (ATSC), which consist of links that bypass several layers to be directly added to the output of later layers (see Fig. REF (b)) [1]}.
