With the rise of end-to-end learning through deep learning, person detectors and re-identification (ReID) models have recently become very strong. Multi-camera multi-target (MCMT) tracking has not fully gone through this transformation yet. We intend to take another step in this direction by presenting a theoretically principled way of integrating ReID with tracking formulated as an optimal Bayes filter. This conveniently side-steps the need for data-association and opens up a direct path from full images to the core of the tracker. While the results are still sub-par, we believe that this new, tight integration opens many interesting research opportunities and leads the way towards full end-to-end tracking from raw pixels.
One of the conceptual differences between the aforementioned works and our explorative work is exactly this dependency on a person detector, providing discrete boxes as starting points. This gives limited state information regarding position, and makes tracking an instance of the complex data association problem. First work towards end-to-end tracking by learning this data association was recently done by @cite_5 . But as mentioned in the introduction, we want to drop both the data association and discrete box representations, and instead keep track of the full belief for each person by leveraging recent ReID models.
@cite_5
2082716591 2048605796 2293909728 2739491435
We address the problem of multi-person tracking in a complex scene from a single camera. Although tracklet-association methods have shown impressive results in several challenging datasets, discriminability of the appearance model remains a limitation. Inspired by the work of person identity recognition, we obtain discriminative appearance-based affinity models by a novel framework to incorporate the merits of person identity recognition, which help multi-person tracking performance. During off-line learning, a small set of local image descriptors is selected to be used in on-line learned appearances-based affinity models effectively and efficiently. Given short but reliable track-lets generated by frame-to-frame association of detection responses, we identify them as query tracklets and gallery tracklets. For each gallery tracklet, a target-specific appearance model is learned from the on-line training samples collected by spatio-temporal constraints. Both gallery tracklets and query tracklets are fed into hierarchical association framework to obtain final tracking results. We evaluate our proposed system on several public datasets and show significant improvements in terms of tracking evaluation metrics. The measurement selection for updating the state estimate of a target's track, known as data association, is essential for good performance in the presence of spurious measurements or clutter. A classification of tracking and data association approaches has been presented, as a pure MMSE approach, which amounts to a soft decision, and single best-hypothesis approach, which amounts to a hard decision. It has been shown that the optimal state estimator in the presence of data association uncertainty consists of the computation of the conditional pdf of the state x(k) given all information available at time k, namely, the prior information about the initial state, the intervening known inputs, and the sets of measurements through time k. It has also been pointed out that if the exact conditional pdf, which is a mixture, is available, then its recursion requires only the probabilities of the most recent association events. The conditions under which this result holds, namely whiteness of the noise, detection, and clutter processes, were presented. The PDAF and JPDAF algorithms, which carry out data association and state estimation in clutter, have been described. A simple example was given to illustrate how the clutter and occasional missed detections can lead to track loss for a standard tracking filter, and how PDAF can keep the target in track under such circumstances. By using the Monte Carlo in a simulated based surveillance as an exampled shown. The numerous applications of the PDAF JPDAF illustrated in "Real-World Applications of PDAF and JPDAF" show the potential pitfalls of using sophisticated algorithms for tracking in difficult environments as well as how to overcome them. The effect of finite sensor resolution can be a more severe problem than the data association and deserves special attention. Multi-person tracking is still a challenging problem due to recurrent occlusion, pose variation and similar appearances between people. Inspired by the success of sparse representations in single object tracking and face recognition, we propose in this paper an online tracking by detection framework based on collaborative sparse representations. We argue that collaborative representations can better differentiate people compared to target-specific models and therefore help to produce a more robust tracking system. We also show that despite the size of the dictionaries involved, these representations can be efficiently computed with large-scale optimization techniques to get a near real-time algorithm. Experiments show that the proposed approach compares well to other recent online tracking systems on various datasets. Tracking multiple persons in a monocular video of a crowded scene is a challenging task. Humans can master it even if they loose track of a person locally by re-identifying the same person based on their appearance. Care must be taken across long distances, as similar-looking persons need not be identical. In this work, we propose a novel graph-based formulation that links and clusters person hypotheses over time by solving an instance of a minimum cost lifted multicut problem. Our model generalizes previous works by introducing a mechanism for adding long-range attractive connections between nodes in the graph without modifying the original set of feasible solutions. This allows us to reward tracks that assign detections of similar appearance to the same person in a way that does not introduce implausible solutions. To effectively match hypotheses over longer temporal gaps we develop new deep architectures for re-identification of people. They combine holistic representations extracted with deep networks and body pose layout obtained with a state-of-the-art pose estimation model. We demonstrate the effectiveness of our formulation by reporting a new state-of-the-art for the MOT16 benchmark. The code and pre-trained models are publicly available.