We examine the problem of joint top-down active search of multiple objects under interaction, e.g., person riding a bicycle, cups held by the table, etc.. Such objects under interaction often can provide contextual cues to each other to facilitate more efficient search. By treating each detector as an agent, we present the first collaborative multi-agent deep reinforcement learning algorithm to learn the optimal policy for joint active object localization, which effectively exploits such beneficial contextual information. We learn inter-agent communication through cross connections with gates between the Q-networks, which is facilitated by a novel multi-agent deep Q-learning algorithm with joint exploitation sampling. We verify our proposed method on multiple object detection benchmarks. Not only does our model help to improve the performance of state-of-the-art active localization models, it also reveals interesting co-detection patterns that are intuitively interpretable.
Another interesting connection relates to works of pose estimation, landmark detection etc., where one is assigned the task of localizing the positions of all joints or landmarks potentially highly related due to physical constraints. Explicit learning the relationship of different joints landmarks has been studied in such literatures @cite_33 @cite_24 . However similar ideas were rarely explored in general localization problems where such interactions are relatively implicit. Our work partially fills the gap here and proves the concept is similarly applicable in many object-level localization problems.
@cite_24 @cite_33
2562512475 2079846689
Modeling the relationship among human joints is one of the most important components in human pose estimation. Most of previous methods define this relationship as a geometric constraint on the relative locations of two neighboring joints. In this constraint, the local appearance of the region connecting two neighboring joints is ignored. However, discarding this image appearance leads to some severe problems, such as double-counting and localization failure when the human pose is rare in the training dataset. Moreover, this image appearance, called human limb, plays an important role in human pose estimation in human visual system. Due to these reasons, we propose to solve a new task: human limb detection, which aims at detecting and representing this local image appearance. We combine this task with human joint localization as a unified framework. After getting the initial detections, we design a two-steps graphical model to capture the spatial relationship among human joints and limbs in a coarse to fine way. We evaluate the proposed method on two widely used datasets for human pose estimation: 1) frame labeled in cinema and 2) leeds sports pose datasets. The experiments results show the effectiveness of our method. We present a novel method for accurate marker-less capture of articulated skeleton motion of several subjects in general scenes, indoors and outdoors, even from input filmed with as few as two cameras. Our approach unites a discriminative image-based joint detection method with a model-based generative motion tracking algorithm through a combined pose optimization energy. The discriminative part-based pose detection method, implemented using Convolutional Networks (ConvNet), estimates unary potentials for each joint of a kinematic skeleton model. These unary potentials are used to probabilistically extract pose constraints for tracking by using weighted sampling from a pose posterior guided by the model. In the final energy, these constraints are combined with an appearance-based model-to-image similarity term. Poses can be computed very efficiently using iterative local optimization, as ConvNet detection is fast, and our formulation yields a combined pose estimation energy with analytic derivatives. In combination, this enables to track full articulated joint angles at state-of-the-art accuracy and temporal stability with a very low number of cameras.