So far, we have analyzed networks able to use cnn features to effectively locate and categorize different objects in an image with bounding boxes. Mask R-CNN [1]} extends such techniques to go one step further and locate exact pixels of each object instead of just bounding boxes (i.e. performing instance segmentation). Mask R-CNN does this by adding a branch to Faster R-CNN that outputs a binary mask or each RoI that determines whether or not a given pixel is part of an object. This branch, as in Faster R-CNN for the rpn, is a fcn on top of a cnn feature map. To avoid the misalignments caused by the original RoIPool layer, an additional RoIAlign layer was proposed to preserve the pixel-level spatial correspondence.
