Metrics on the space of sets of trajectories are important for scientists in the field of computer vision, machine learning, robotics, and general artificial intelligence. However, existing notions of closeness between sets of trajectories are either mathematically inconsistent or of limited practical use. In this paper, we outline the limitations in the current mathematically-consistent metrics, which are based on OSPA ( 2008); and the inconsistencies in the heuristic notions of closeness used in practice, whose main ideas are common to the CLEAR MOT measures (Keni and Rainer 2008) widely used in computer vision. In two steps, we then propose a new intuitive metric between sets of trajectories and address these limitations. First, we explain a solution that leads to a metric that is hard to compute. Then we modify this formulation to obtain a metric that is easy to compute while keeping the useful properties of the previous metric. Our notion of closeness is the first demonstrating the following three features: the metric 1) can be quickly computed, 2) incorporates confusion of trajectories' identity in an optimal way, and 3) is a metric in the mathematical sense.
One of the reasons why the CLEAR MOT metrics are widely used is because they create a simple association between @math and @math , i.e., the association between @math and @math does not change often in time. It appears that @cite_44 is one of the first works that describes how to control the number of association changes when computing a similarity measure. The authors do not associate @math and @math independently every time instant but rather use a sequential matching procedure that tries to keep the association from the previous time instant if possible. This is similar to the procedure used in the much more recent CLEAR MOT measures that we discuss in the Introduction.
@cite_44
1563723533
This paper presents a new formalized procedure to assess the performance of automatic target tracking systems. The procedure involves comparing data output by a tracking system under test with track data in a truth file. The comparison of the data sources uses a total of 15 tracker performance metrics. These cover the categories of track initiation, track maintenance, track error and false tracks. The performance for each metric is expressed as a probability. An overall performance measure comes from using a weighted combination of these probabilities. The weighting relates to the metric’s importance. This approach is embodied in a Tracker Assessment Tool (TAT) and the processing steps in the TAT are reviewed with an example. A method to simplify the creation of truth data from recorded data is also described.