Even with all the advancements observed in Fast R-CNN, there was still one remaining bottleneck in the process: the region proposer. As explained above, the first step of the detection pipeline is to generate a bunch of potential bounding boxes or Regions of Interest likely to contain objects. As we saw, in R-CNN and Fast R-CNN, these proposals were created using the Selective Search algorithm [1]}, a fairly slow process that was found to be the bottleneck of the overall process. In 2015, a team of Microsoft research proposed in [2]} a further evolution of R-CNN and Fast R-CNN, named Faster R-CNN, finding a way to make the region proposal step more efficient. In particular, they proposed to compute the region proposals exploiting the features of the images extracted by the cnn that acts as the backbone in Fast R-CNN. In other words, they suggested using the extracted features of the images not only for the classification and bounding boxes regression steps of the detection pipeline but also for the region proposals computation instead of running the slow and costly selective search algorithm. More in detail, Faster R-CNN introduces a new network, the rpn, that analyzes the features of the images, ranking region boxes (called anchors), and proposes the ones most likely containing objects. Basically, the authors proposed a single network comprising rpn for region proposal and Fast RCNN for region classification. rpn and Fast RCNN share a large number of convolutional layers and, in particular, the features from the last shared convolutional layer are used for region proposal and region classification from separate branches, thus enabling highly efficient region proposal computation. rpn is, in fact, a fcn; Faster RCNN is thus a purely cnn based framework without using handcrafted features.
