Stereo matching is one of the important basic tasks in the computer vision field. In recent years, stereo matching algorithms based on deep learning have achieved excellent performance and become the mainstream research direction. Existing algorithms generally use deep convolutional neural networks (DCNNs) to extract more abstract semantic information, but we believe that the detailed information of the spatial structure is more important for stereo matching tasks. Based on this point of view, this paper proposes a shallow feature extraction network with a large receptive field. The network consists of three parts: a primary feature extraction module, an atrous spatial pyramid pooling (ASPP) module and a feature fusion module. The primary feature extraction network contains only three convolution layers. This network utilizes the basic feature extraction ability of the shallow network to extract and retain the detailed information of the spatial structure. In this paper, the dilated convolution and atrous spatial pyramid pooling (ASPP) module is introduced to increase the size of receptive field. In addition, a feature fusion module is designed, which integrates the feature maps with multiscale receptive fields and mutually complements the feature information of different scales. We replaced the feature extraction part of the existing stereo matching algorithms with our shallow feature extraction network, and achieved state-of-the-art performance on the KITTI 2015 dataset. Compared with the reference network, the number of parameters is reduced by 42%, and the matching accuracy is improved by 1.9%. Since the introduction of deep learning in the computer vision field, increasing the network depth (that is, the number of layers in the network) seems to be a necessary means to improve the feature extraction ability. Taking the object classification task as an example, as the network depth increases from the 8-layer network AlexNet (Krizhevsky et al., 2012) to the 16-layer network VGG (Simonyan & Zisserman, 2014) and to the 101-layer network ResNet (He et al., 2015) , the classification accuracy constantly improves. There are two purposes of the deep network. First, the deep network can improve the ability to extract abstract features (Zeiler & Fergus, 2013) , which are important for some vision tasks, such as object detection (Girshick, 2015; Ren et al., 2017) and classification. For example, for objects such as cups, their colors, shapes and sizes may be different, and they cannot be accurately identified using only these primary feature information. Therefore, the feature extraction network must have the ability to extract more abstract semantic information. Second, the deep feature extraction network can obtain a larger receptive field to learn more context information (Luo et al., 2017; Liu et al., 2018) . With the increase in the number of network layers, the size of the receptive field is also constantly increasing. In particular, after image sampling using a pooling operation, even the 3*3 convolution kernel has the ability to extract context information. Many studies (Zeiler & Fergus, 2013; Yu & Koltun, 2016) have shown that the lower part of the convolution neural network mainly extracts primary features, such as the edges and corners, while the higher part can extract more abstract semantic information. However, many basic vision tasks rely more on basic feature information instead of the high-level abstract features. Stereo matching is one of the basic vision tasks. In the traditional stereo matching algorithm (Scharstein & Szeliski, 2002) , the color similarity metrics of pixels are usually used to calculate the matching costs between the left and right images to find the matching points in the two images. After the introduction of deep learning, more robust feature information can be obtained through training and learning, which can effectively improve the performance of the stereo matching algorithm. At present, many excellent stereo matching algorithms based on deep learning, such as the GC-Net (Kendall et al., 2017) , PSMNet (Chang & Chen, 2018) and GwcNet (Guo et al., 2019) , generally adopt similar processes, including feature extraction, matching cost volume construction, 3D convolution and disparity regression. This paper focuses on the feature extraction steps. The stereo matching task has two requirements for the feature extraction network. The first requirement is the enlargement of the receptive field as far as possible so that the network can obtain more context information, which is critical to solving the mismatching problems in the discontinuous disparity area. Because a larger receptive field can learn the relationships between different objects, even if there are problems, such as conclusion or inconsistent illumination, the network can use the context information to infer disparity and improve the stereo matching accuracy in the ill-posed regions. The second requirement is the maintenance of more details of the spatial structure, which can improve the matching accuracy of many small structures, such as railings, chains, traffic signs and so on. The existing feature extraction networks usually use a deep convolution neural network to obtain a larger receptive field and extract more abstract semantic information. In this process, with the increase of the network layers and the compression of the image size, substantial detailed information of the spatial structure is inevitably lost. We believe that compared with the abstract semantic information that is extracted by a deep network, the detailed information of the spatial structure is more important to improving the stereo matching accuracy. Based on this point of view, this paper proposes a novel structure of feature extraction network -a shallow feature extraction network. Unlike the common feature extraction network (with ResNet-50 as the backbone), in this paper, the backbone of the feature extraction network only has 3 convolution layers, and the image is only downsampled once in the first convolution layer to compress the size of the image. This structure retains more details of the spatial structure and pays more attention to primary features such as the edges and corners of objects, while abandoning more abstract semantic information. To solve the problem that the size of the receptive field of the shallow structure is limited, this paper introduces the atrous spatial pyramid pooling (ASPP) module . The ASPP module uses the dilated convolution to increase the receptive field size without increasing the number of parameters. In addition, the convolution layers with different dilation rate can obtain feature maps with multiscale receptive fields. The large receptive fields can be used to obtain context information and to solve the problem of mismatching in ill-posed regions, and the small receptive fields can be used to retain more detailed information of the spatial structure and to improve the stereo matching accuracy in local areas. To integrate feature maps with multiscale receptive fields, this paper designs the feature fusion module and introduces the channel attention mechanism (Jie et al., 2017) . We assign different weights to feature maps with different dilation rates in the channel dimensions. The weights are acquired through learning, and more weight and attention are given to the feature channels with greater roles. The advantages of a shallow feature extraction network with a large receptive field are twofold. One advantage is that the network can meet the two requirements of the stereo matching task for the feature extraction network. On the basis of ensuring the large receptive field, more details of the spatial structure are retained. The other advantage is that the network greatly reduces the number of parameters and the difficulties of network training and deployment. The feature extraction network that is designed in this paper is used to replace the feature extraction part of the existing stereo matching network, and state-of-the-art performance is achieved on the KITTI2015 dataset (Geiger, 2012) . Compared with the reference network, the number of parameters is reduced by 42%, and the matching accuracy is improved by 1.9%. The main contributions of this paper are as follows. • A shallow feature extraction network is proposed to extract and retain more details of the spatial structure. This network can improve the stereo matching accuracy with fewer parameters. • The dilated convolution and ASPP module are introduced to enlarge the receptive field. We verify the effect of the dilated convolution on the receptive field using mathematics and experiments. • A feature fusion module, which integrates the feature maps with multiscale receptive fields, is designed and realizes the mutual complementary feature information of different scales. Focusing on the feature extraction part of a stereo matching network, this paper proposes a novel network structure, which abandons the popular deep convolution neural network and use the shallow network structure to extract and retain more basic feature information. To solve the problem that the receptive field of a shallow network is limited, this paper introduces the ASPP module and obtains multiscale receptive fields by adding convolution branches with different dilation rates. By using the feature fusion module, the feature maps with multiscale receptive fields are fused together to solve the information loss problem that is caused by dilated convolution. Finally, a large and dense receptive field is obtained. The shallow feature extraction network with a large receptive field can provide more suitable feature information for stereo matching task, with fewer parameters and lower training difficulty. Using the SWNet to replace the feature extraction part of the existing network can effectively improve the stereo matching accuracy. A APPENDIX Figure 4 : Schematic diagram of neurons corresponding to receptive fields. To clearly explain the calculation process of the theoretical receptive field and effective receptive field, the 2D convolution neural network is simplified into a 1D neural network similar to multilayer perceptron (MLP). The connection relationship between its neurons is shown in Figure 4 , where each circle represents one neuron. Limited by the size of the image, only half of the receptive field of the neuron is shown. The receptive field of the neuron in layer 0 (input layer) is 1, that is r 0 = 1. The receptive field of the neuron in layer 1 is r 1 = r 0 × k 1 = 1 × 3 = 3. The receptive field of neurons in layer 2 is r 2 = r 1 × k 2 = 3 × 3 = 9 , but since neurons are not independent of each other, there are overlaps between their receptive fields, so the overlaps must be subtracted when calculating the size of the receptive field. The number of neurons in the overlapping part is related to the kernel size and the convolution stride. As shown in Figure 4 , the kernel size of the neurons in layer 2 is three. Then there are two overlaps in the corresponding receptive field, and the number of neurons that is contained in each overlaps is one. Therefore, the number of neurons that is contained in all overlaps is as follows. Then the size of receptive field of neuron in layer 2 should be modified as It is worth noting that, in the convolution neural network, as the number of convolution layers increases, the impact of convolution stride is cumulative. Therefore, the size of the receptive field of the neuron in layer n should be formulated as For dilated convolution, the kernel size should be modified as By substituting formula (10) into formula (9), the size of the theoretical receptive field of the dilated convolution can be calculated as For the size of the effective receptive field, this paper only studies the case when the convolution stride is smaller than the kernel size, which is k n > s n . As shown in Figure 4 , the kernel of the neuron in layer 3 is dilated, and the information of some low-level neurons will not be transmitted to the neuron in layer 3, which are called invalid neurons (black circles in Figure 4 ). The maximum number of continuous invalid neurons in layer 2 is the dilation rate of layer 3 minus 1, which is p 2 = d 3 − 1 = 5 − 1 = 4 . The maximum number of continuously invalid neurons in layer 0-1 is related to the connection relationship between network layers. To describe this relationship, this paper introduces the concepts of exclusive subneurons and shared subneurons. Subneurons refer to the low-level neurons that are directly connected to the neurons in higher layers. As shown in Figure 4 , the green neurons are the subneurons of purple neurons, while the black neurons are not. An exclusive subneuron refers to the only sub-neuron in layer (n-1) that is connected to a neuron in layer n. As shown in Figure 4 , the red neurons are the exclusive subneurons of the yellow neurons. Under the 1D condition, each neuron has two adjacent neurons, and there is overlap between the subneurons of every two neurons. Therefore, the number of exclusive subneurons of a neuron in layer n can be calculated as However, the number of exclusive subneurons should be non-negative, with a minimum value of 0. Therefore, a non-negative constraint is added to formula (12) Therefore, if one neuron in layer n fails, it will directly lead to the failure of N n subneurons in layer (n-1). A shared subneuron refers to the subneuron that is connected with multiple neurons in higher layers. As shown in Figure 4 , the blue neurons are the shared neurons of the yellow neurons. A shared subneuron in layer (n-1) is connected to M n neurons in layer n. In other words, if there are M n continuously invalid neurons in layer n, there will be one invalid neuron in layer (n-1). The calculation method of M n is M n = k n − s n + 1 Comprehensively considering the exclusive subneurons and shared subneurons, when there are p n invalid neurons in layer n, the number of invalid neurons in layer (n-1) is p n−1 = p n N n + (p n − M n + 1) = p n (N n + 1) − M n + 1 If the invalid neuron in layer n is directly caused by the dilated convolution, the number of invalid neurons in layer n is p n = d n+1 − 1 As shown in Figure 4 , the number of invalid neurons in layer 2 is p 2 = d 3 − 1 = 5 − 1 = 4 . The numbers of invalid neurons in layer 1 and 0 are p 1 = 4 × (0 + 1) − 3 + 1 = 2 and p 0 = 2 × (1 + 1) − 2 + 1 = 3, respectively. The size of the effective receptive field should be the size of theoretical receptive field minus the number of invalid neurons in layer 0. The calculation method is shown in formula (17) r n = r n − p 0 (k n − 1) B APPENDIX K denotes the convolution kernel size, C denotes the number of output channels, S denotes the convolution stride, D denotes the dilation rate, BN denotes the batch normalization layer, ReLU denotes the activation layer, H denotes the height of the image and W denotes the width of the image. Concat stands for the concatenation operation of feature maps, and SElayer stands for assigning weights to each feature map.
We introduced a shallow featrue extraction network with a large receptive field for stereo matching tasks, which uses a simple structure to get better performance.
{"src": "scitldr-AIC_train", "paper_id": "H1lKNp4Fvr"}