To count the vehicles, we exploited a detection-based approach since regression-based techniques cannot precisely localize the objects present in the scene and can eventually provide only a coarse position of the area in which they are distributed (see also sec:back:visual-counting for further details), therefore resulting not suitable in our scenario where we need to know the precise localization (with boundaries) of the detected vehicles. As in the previous sec:counting-on-the-edge:counting-drones where we proposed a solution able to estimate the number of vehicles in uav imagery, also in this research, we faced challenges due to the imitated computational resources available in smart cameras. Here, we relied on Mask R-CNN [1]}, a popular deep CNN for instance segmentation that operates within the `recognition using regions paradigm [2]}, described in sec:back:cnn-based-detectors:two-stages-detectors. In particular, it extends the Faster R-CNN detector [3]} by adding a branch that outputs a binary mask indicating whether or not a given pixel is part of an object. Briefly, in the first stage, a cnn acts as a backbone, extracting the input image features. Starting from this feature space, another cnn named rpn generates region proposals that might contain objects. rpn slices pre-defined region boxes (called anchors) over this space and ranks them, suggesting those most likely containing objects. Once rpn produces the Regions of Interested (ROIs), they might be of different sizes. Since it is hard to work on features having different sizes, rpn reduces them into the same dimension using the Region of Interest Pooling algorithm. Finally, these fixed-size proposals are processed by two parallel CNN-based branches: one is responsible for classifying and localizing the objects inside them with bounding boxes; the second produces a binary mask that says whether or not a given pixel is part of an object. In the end, given an input image, the network produces per-pixel masks localizing the detected objects together with the associated labels classifying them. To make our counting solution able to run efficiently directly on the devices, we employed, as a backbone, the ResNet50 architecture, a lighter version of the popular ResNet101 [4]}. This simplification is also justified because the more powerful version of Mask R-CNN based on the ResNet101 model was designed for more complicated visual detection tasks than ours. Originally, Mask R-CNN was trained on the COCO dataset [5]} to detect and recognize 80 different classes of everyday objects. In our case, we have to localize and identify objects belonging to just one category (i.e., the vehicle category). To this end, we further simplified the model by reducing the number of the final fully convolutional layers responsible for the classification of the detected objects, making the model lighter. Once we have localized the instances of the objects, we count them estimating the number of vehicles present in the scene.
