Across a majority of pedestrian detection datasets, it is typically assumed that pedestrians will be standing upright with respect to the image coordinate system. This assumption, however, is not always valid for many vision-equipped mobile platforms such as mobile phones, UAVs or construction vehicles on rugged terrain. In these situations, the motion of the camera can cause images of pedestrians to be captured at extreme angles. This can lead to very poor pedestrian detection performance when using standard pedestrian detectors. To address this issue, we propose a Rotational Rectification Network (R2N) that can be inserted into any CNN-based pedestrian (or object) detector to adapt it to significant changes in camera rotation. The rotational rectification network uses a 2D rotation estimation module that passes rotational information to a spatial transformer network to undistort image features. To enable robust rotation estimation, we propose a Global Polar Pooling (GP-Pooling) operator to capture rotational shifts in convolutional features. Through our experiments, we show how our rotational rectification network can be used to improve the performance of the state-of-the-art pedestrian detector under heavy image rotation by up to 45
@cite_14 introduce a general warp framework called Spatial Transformer Networks to enable affine transformations with differentiable sampling inside the network. It achieves transformation invariance within CNN architectures very efficiently without data augmentation. An important point which is often overlooked is that the design of the whose purpose is to estimate transformation parameters is not explored in the original work. They use two baseline CNNs as the localization networks in the spatial transformer: (1) two fully-connected layers; (2) a CNN with two convolutional and two max pooling layers. The models are evaluated on the distorted MNIST dataset. However, the MNIST dataset is small and has low resolution images which is not a strict criteria to judge transformation invariance of a network. In other words, the design of a network with natural transformation invariance is still an open problem. Note that our work is complementary to the spatial transformer because our proposed rotation estimation module with GP-Pooling operators can be viewed as an expert localization network with natural rotation invariance.
@cite_14
603908379
Convolutional Neural Networks define an exceptionally powerful class of models, but are still limited by the lack of ability to be spatially invariant to the input data in a computationally and parameter efficient manner. In this work we introduce a new learnable module, the Spatial Transformer, which explicitly allows the spatial manipulation of data within the network. This differentiable module can be inserted into existing convolutional architectures, giving neural networks the ability to actively spatially transform feature maps, conditional on the feature map itself, without any extra training supervision or modification to the optimisation process. We show that the use of spatial transformers results in models which learn invariance to translation, scale, rotation and more generic warping, resulting in state-of-the-art performance on several benchmarks, and for a number of classes of transformations.