With the advent of deep learning, object detection drifted from a bottom-up to a top-down recognition problem. State of the art algorithms enumerate a near-exhaustive list of object locations and classify each into: object or not. In this paper, we show that bottom-up approaches still perform competitively. We detect four extreme points (top-most, left-most, bottom-most, right-most) and one center point of objects using a standard keypoint estimation network. We group the five keypoints into a bounding box if they are geometrically aligned. Object detection is then a purely appearance-based keypoint estimation problem, without region classification or implicit feature learning. The proposed method performs on-par with the state-of-the-art region based detection methods, with a bounding box AP of 43.2 on COCO test-dev. In addition, our estimated extreme points directly span a coarse octagonal mask, with a COCO Mask AP of 18.9 , much better than the Mask AP of vanilla bounding boxes. Extreme point guided segmentation further improves this to 34.6 Mask AP.
Prevalent keypoint detection methods work on well-defined semantic keypoints, e.g., human joints. StarMap @cite_49 mixes all types of keypoints using a single heatmap for general keypoint detection. Our extreme and center points are a kind of such general implicit keypoints, but with more explicit geometry property.
@cite_49
2795096917 2589255576 1945298332 2949359214
Semantic keypoints provide concise abstractions for a variety of visual understanding tasks. Existing methods define semantic keypoints separately for each category with a fixed number of semantic labels in fixed indices. As a result, this keypoint representation is in-feasible when objects have a varying number of parts, e.g. chairs with varying number of legs. We propose a category-agnostic keypoint representation, which combines a multi-peak heatmap (StarMap) for all the keypoints and their corresponding features as 3D locations in the canonical viewpoint (CanViewFeature) defined for each instance. Our intuition is that the 3D locations of the keypoints in canonical object views contain rich semantic and compositional information. Using our flexible representation, we demonstrate competitive performance in keypoint detection and localization compared to category-specific state-of-the-art methods. Moreover, we show that when augmented with an additional depth channel (DepthMap) to lift the 2D keypoints to 3D, our representation can achieve state-of-the-art results in viewpoint estimation. Finally, we show that our category-agnostic keypoint representation can be generalized to novel categories. Keypoint detection is one of the most importantpre-processing steps in tasks such as face modeling, recognitionand verification. In this paper, we present an iterative methodfor Keypoint Estimation and Pose prediction of unconstrainedfaces by Learning Efficient H-CNN Regressors (KEPLER) foraddressing the face alignment problem. Recent state of the artmethods have shown improvements in face keypoint detectionby employing Convolution Neural Networks (CNNs). Althougha simple feed forward neural network can learn the mappingbetween input and output spaces, it cannot learn the inherentstructural dependencies. We present a novel architecture calledH-CNN (Heatmap-CNN) which captures structured global andlocal features and thus favors accurate keypoint detecion. H-CNNis jointly trained on the visibility, fiducials and 3D-pose of theface. As the iterations proceed, the error decreases making thegradients small and thus requiring efficient training of DCNNs tomitigate this. KEPLER performs global corrections in pose andfiducials for the first four iterations followed by local correctionsin a subsequent stage. As a by-product, KEPLER also provides3D pose (pitch, yaw and roll) of the face accurately. In thispaper, we show that without using any 3D information, KEPLERoutperforms state of the art methods for alignment on challengingdatasets such as AFW [38] and AFLW [17]. We introduce a learning-based approach to detect repeatable keypoints under drastic imaging changes of weather and lighting conditions to which state-of-the-art keypoint detectors are surprisingly sensitive. We first identify good keypoint candidates in multiple training images taken from the same viewpoint. We then train a regressor to predict a score map whose maxima are those points so that they can be found by simple non-maximum suppression. As there are no standard datasets to test the influence of these kinds of changes, we created our own, which we will make publicly available. We will show that our method significantly outperforms the state-of-the-art methods in such challenging conditions, while still achieving state-of-the-art performance on untrained standard datasets. We propose a method for multi-person detection and 2-D pose estimation that achieves state-of-art results on the challenging COCO keypoints task. It is a simple, yet powerful, top-down approach consisting of two stages. In the first stage, we predict the location and scale of boxes which are likely to contain people; for this we use the Faster RCNN detector. In the second stage, we estimate the keypoints of the person potentially contained in each proposed bounding box. For each keypoint type we predict dense heatmaps and offsets using a fully convolutional ResNet. To combine these outputs we introduce a novel aggregation procedure to obtain highly localized keypoint predictions. We also use a novel form of keypoint-based Non-Maximum-Suppression (NMS), instead of the cruder box-level NMS, and a novel form of keypoint-based confidence score estimation, instead of box-level scoring. Trained on COCO data alone, our final system achieves average precision of 0.649 on the COCO test-dev set and the 0.643 test-standard sets, outperforming the winner of the 2016 COCO keypoints challenge and other recent state-of-art. Further, by using additional in-house labeled data we obtain an even higher average precision of 0.685 on the test-dev set and 0.673 on the test-standard set, more than 5 absolute improvement compared to the previous best performing method on the same dataset.