In the recent years, a number of novel, deep-learning based, interest point detectors, such as LIFT, DELF, Superpoint or LF-Net was proposed. However there’s a lack of a standard benchmark to evaluate suitability of these novel keypoint detectors for real-live applications such as autonomous driving. Traditional benchmarks (e.g. Oxford VGG) are rather limited, as they consist of relatively few images of mostly planar scenes taken in favourable conditions. In this paper we verify if the recent, deep-learning based interest point detectors have the advantage over the traditional, hand-crafted keypoint detectors. To this end, we evaluate stability of a number of hand crafted and recent, learning-based interest point detectors on the street-level view ApolloScape dataset.
Quad-networks @cite_22 is the first unsupervised interest point detection method, not relying on a hand crafted feature detector for the training set generation. Neural network is trained to map an image patch point to a single real-valued response and then to rank points according to the response. Image patch is mapped by the neural network to a 'heat map' with ranking of each pixel. The ranking is optimized to be repeatable under the desired transformation classes, such as rotation, translation or intensity change. If one point has the higher ranking than another one, it should still be higher after a transformation. Top and bottom quantiles of the response are repeatable and can be used as an interest points. Two training approaches are described. The first one uses an image dataset with ground truth data on camera poses and 3D scene structure from the LIDAR scan. Using the ground truth data, correspondence between image patches in two different images can be established. The other approach is fully unsupervised. The training set is constructed, by applying transformations, such as rotation and translation, to a set of base image patches.
@cite_22
2556970001
Several machine learning tasks require to represent the data using only a sparse set of interest points. An ideal detector is able to find the corresponding interest points even if the data undergo a transformation typical for a given domain. Since the task is of high practical interest in computer vision, many hand-crafted solutions were proposed. In this paper, we ask a fundamental question: can we learn such detectors from scratch? Since it is often unclear what points are interesting, human labelling cannot be used to find a truly unbiased solution. Therefore, the task requires an unsupervised formulation. We are the first to propose such a formulation: training a neural network to rank points in a transformation-invariant manner. Interest points are then extracted from the top bottom quantiles of this ranking. We validate our approach on two tasks: standard RGB image interest point detection and challenging cross-modal interest point detection between RGB and depth images. We quantitatively show that our unsupervised method performs better or on-par with baselines.