Metrics on the space of sets of trajectories are important for scientists in the field of computer vision, machine learning, robotics, and general artificial intelligence. However, existing notions of closeness between sets of trajectories are either mathematically inconsistent or of limited practical use. In this paper, we outline the limitations in the current mathematically-consistent metrics, which are based on OSPA ( 2008); and the inconsistencies in the heuristic notions of closeness used in practice, whose main ideas are common to the CLEAR MOT measures (Keni and Rainer 2008) widely used in computer vision. In two steps, we then propose a new intuitive metric between sets of trajectories and address these limitations. First, we explain a solution that leads to a metric that is hard to compute. Then we modify this formulation to obtain a metric that is easy to compute while keeping the useful properties of the previous metric. Our notion of closeness is the first demonstrating the following three features: the metric 1) can be quickly computed, 2) incorporates confusion of trajectories' identity in an optimal way, and 3) is a metric in the mathematical sense.
It is worth mentioning a few works that differ from the main stream in this aspect. One of them is @cite_1 , where the authors define a similarity measure based on comparing the occurrence of special discrete events in @math and @math , and another is @cite_30 , where the authors propose an information theoretic measure of similarity between sets of trajectories. Finally, in @cite_7 the authors propose a similarity measured based on hidden Markov models that does not assume that the temporal sampling rates of the trajectories are equal.
@cite_30 @cite_1 @cite_7
1581210256 2130827092 2140017549
Automated tracking of vehicles and people is essential for the effective utilization of imagery in wide area surveillance applications. In order to determine the best tracking algorithm and parameters for a given application, a comprehensive evaluation procedure is required. However, despite half a century of research in multi-target tracking, there is no consensus on how to score the overall performance of these trackers. Existing evaluation approaches assess tracker performance through measures of correspondence between ground truth tracks and system tracks using metrics such as track detection rate, track completeness, track fragmentation rate, and track ID change rate. However, each of these only provides a partial measure of performance and no good method exists to combine them into a holistic metric. Towards this end, this paper presents a pair of information theoretic metrics with similar behavior to the Receiver Operating Characteristic (ROC) curves of signal detection theory. Overall performance is evaluated with the percentage of truth information that a tracker captured and the total amount of false information that it reported. Information content is quantified through conditional entropy and mutual information computations using numerical estimates of the probability of association between the truth and the system tracks. This paper demonstrates how these information quality metrics provide a comprehensive evaluation of overall tracker performance and how they can be used to perform tracker comparisons and parameter tuning on wide-area surveillance imagery and other applications.1 A tracking system outputs a separate motion trajectory for each moving object in a scene. The paper presents a problem of performance evaluation and performance metrics for real time systems that track people, or moving objects, in video sequences, and it proposes performance measurement methodology for such systems. Two approaches to measuring performance are presented. The first approach compares the computed motion trajectories to the reference trajectories. It enables a complete evaluation of tracking results, but reference trajectories it requires are difficult to get. The second, more practical approach identifies in the computed trajectories specific discrete events, such as line crossings, and compares sequences of these events to sequences of reference events, which are much easier to obtain than reference trajectories. These events can usually be chosen such that they reflect the application goal of a tracking system, e.g. counting people in an area. Precision of evaluation increases with density of events. Short event sequences measure the sensitivity and selectivity of a tracking method, i.e. how well it satisfies the "one person one trajectory" objective. Long sequences measure continuity of trajectories: how long a method can keep track of one person. The paper shows performance measurement results for a real time people tracking system developed by the authors. In this paper, we introduce a set of novel distance metrics that use model based representations for trajectories. We determine the similarity of trajectories using the conformity of the corresponding HMM models. These metrics enable the comparison of tracjectories without any limitations of the conventional measures. They accurately identify the coordinate, orientation, and speed affinity. The proposed HMM based distance metrics can be used not only for ground truth comparisons but for clustering as well. Our experiments prove that they have superior discriminative properties.