Initial Left Image R ight Image Hand Designed Cost Aggregation Disparity Map Disparity Computation A ggregated C ost Volume Extracting Guidance Aggregation Guidance Generaing Aggregation Proposals Aggregation Proposals Fusion Aggregated Cost Volume Disparity Map Disparity Computation A The Typical
Deep neural networks have shown excellent performance for stereo matching. Many efforts focus on the feature extraction and similarity measurement of the matching cost computation step while less attention is paid on cost aggregation which is crucial for stereo matching. In this paper, we present a learning based cost aggregation method for stereo matching by a novel sub architecture in the end to end trainable pipeline. We reformulate the cost aggregation as a learning process of the generation and selection of cost aggregation proposals which indicate the possible cost aggregation results. The cost aggregation sub architecture is realized by a two stream network: one for the generation of cost aggregation proposals, the other for the selection of the proposals. The criterion for the selection is determined by the low level structure information obtained from a light convolutional network. The two stream network offers a global view guidance for the cost aggregation to rectify the mismatching value stemming from the limited view of the matching cost computation. The comprehensive experiments on challenge datasets such as KITTI and Scene Flow show that our method outperforms the state of the art methods.