Metrics on the space of sets of trajectories are important for scientists in the field of computer vision, machine learning, robotics, and general artificial intelligence. However, existing notions of closeness between sets of trajectories are either mathematically inconsistent or of limited practical use. In this paper, we outline the limitations in the current mathematically-consistent metrics, which are based on OSPA ( 2008); and the inconsistencies in the heuristic notions of closeness used in practice, whose main ideas are common to the CLEAR MOT measures (Keni and Rainer 2008) widely used in computer vision. In two steps, we then propose a new intuitive metric between sets of trajectories and address these limitations. First, we explain a solution that leads to a metric that is hard to compute. Then we modify this formulation to obtain a metric that is easy to compute while keeping the useful properties of the previous metric. Our notion of closeness is the first demonstrating the following three features: the metric 1) can be quickly computed, 2) incorporates confusion of trajectories' identity in an optimal way, and 3) is a metric in the mathematical sense.
Some extensions to OSPA incorporate the uncertainty in the measurements. The Q-OSPA metric defined in @cite_15 incorporates uncertainty by weighting the distance between pairs of points by the product of their certainty and by adding a new term that is proportional to the product of the uncertainties. The H-OSPA metric defined in @cite_33 incorporates uncertainty by using OSAP with distributions as elements of @math and @math instead of vectors and using the Hellinger distance between distributions instead of the Euclidean distance between vectors. The authors of both works focus only on the simpler case where the sets @math and @math contain vectors points and not trajectories. However, combining their work with that of @cite_41 or @cite_34 to obtain a metric between sets of trajectories is immediate.
@cite_41 @cite_15 @cite_34 @cite_33
2101295974 2037300854 1881351112 2153363506
Performance evaluation of multi-target tracking algorithms is of great practical importance in the design, parameter optimization and comparison of tracking systems. The goal of performance evaluation is to measure the distance between two sets of tracks: the ground truth tracks and the set of estimated tracks. This paper proposes a mathematically rigorous metric for this purpose. The basis of the proposed distance measure is the recently formulated consistent metric for performance evaluation of multi-target filters, referred to as the OSPA metric. Multi-target filters sequentially estimate the number of targets and their position in the state space. The OSPA metric is therefore defined on the space of finite sets of vectors. The distinction between filtering and tracking is that tracking algorithms output tracks and a track represents a labeled temporal sequence of state estimates, associated with the same target. The metric proposed in this paper is therefore defined on the space of finite sets of tracks and incorporates the labeling error. Numerical examples demonstrate that the proposed metric behaves in a manner consistent with our expectations. The track qualities of multitarget state estimates are often available in many tracking algorithms, including the multiple hypothesis tracking and the joint integrated probabilistic data association estimators. However, the recently proposed Optimal Subpattern Assignment (OSPA) metric ignores the quality information and thus its capability to quantify the performance of multi-object estimation algorithms is limited. In this paper, a new metric, called the quality-based OSPA (Q-OSPA), is proposed based on the original OSPA metric. The proposed Q-OSPA metric is able to incorporate the quality information and thus provide more accurate quantification of the performance of multi-object estimation algorithms. Also, the mathematical consistency of the original OSPA metric is maintained by the proposed Q-OSPA metric. In addition, if the qualities of estimates are not available, the Q-OSPA metric reduces to the original OSPA metric by assigning equal qualities to the estimates. Besides theoretical derivations, simulations are presented to verify the advantages of the proposed metric. This paper proposes a new metric based on the Optimal Sub-pattern Assignment (OSPA) metric for evaluating the performance of multiple target tracking algorithms. It is shown in this paper that by considering the properties of false tracks, missing tracks and miss-detections, the minimization of all distances between tracks enables performance evaluation of multi-target algorithms in a more comprehensive manner than is possible with the OSPA metric for track (OSPAT) introduced by This paper proposes the use of the Hellinger distance in evaluating the localisation error in the OSPA metric. The Hellinger distance provides a measure of the difference between two distributions and is used here to measure the difference between the true and estimated targets where the true and estimated single-target states are characterised by Gaussian distributions. The OSPA metric is used to evaluate the performance of several multisensor PHD and CPHD filters. The importance of introducing track covariance into the metric is demonstrated through the application on several multisensor PHD and CPHD filters. In particular, we are able to identify filters that provide a poor estimate of the uncertainty associated with each track.