The rapid development of convolutional neural networks (CNNs) [1]}, [2]} in the past ten years has greatly promoted various research fields of deep learning, such as image [3]}, [4]}, [5]}, time series [6]}, video [7]}, [8]}, audio [9]}, text [10]}, natural language [11]} and generative adversarial networks [12]}, [13]}. CNNs became the backbone because of their powerful representation ability. With the goal of enhancing the representation of CNNs, researchers have put in a lot of effort and obtained remarkable achievements. Based on the factors that affect the representation ability of CNNs, the existing methods can be divided into quantity-based and quality-based methods. Quantity-based methods can be viewed as “stacking”. For example, width, depth, and cardinality are the three main factors that affect CNNs' performance. Stacking by increasing the width [14]}, [15]}, which means increasing the number of feature maps in a convolutional layer, is a kind of method to further reinforce the performance of the entire network. Expanding depth [16]} of CNNs by stacking more convolutional layers, which is to extract high-dimensional and abstract features, is another method to enhance the representation ability. Different from the increase in width and depth, the increase in cardinality [17]} broadens the convolution operations of a convolutional layer to enrich features. Hence, the cardinality can be seen as the third dimension of quantity-based methods to improve CNNs in addition to width and depth. Quality-based methods are the upgrading of original CNNs' components or the creation of new ones which are plug-and-play. For example, skip connection [18]}, [19]} breaks the bottleneck of direct connection network design, so enables interactive fusion of features between different layers, and strengthens the features' propagation to enhance the representation. Another quality-based method focuses on designing new convolution operations, such as depth separable convolution [20]}, [21]}, dilated convolution [22]}, etc.
