A survey of datasets for visual tracking <s> Datasets for visual tracking <s> Although there is much research on behaviour recognition in time-varying video, there are few ground truthed datasets for assessing multi-person behavioral interactions. This short paper presents the BEHAVE project’s dataset, which has around 90,000 frames of humans identified by bounding boxes, with interacting groups classified into one of 5 different behaviors. An example of its use is also presented. <s> BIB001 </s> A survey of datasets for visual tracking <s> Datasets for visual tracking <s> In this paper, we demonstrate how the differential Earth Mover's Distance (EMD) may be used for visual tracking in synergy with Gaussian mixtures models (GMM). According to our model, motion between adjacent frames results in variations of the mixing proportions of the Gaussian components representing the object to be tracked. These variations are computed in closed form by minimizing the differential EMD between Gaussian mixtures, yielding a very fast algorithm with high accuracy, without recurring to the EM algorithm in each frame. Moreover, we also propose a framework to handle occlusions, where the prediction for the object's location is forwarded to an adaptive Kalman filter whose parameters are estimated on line by the motion model already observed. Experimental results show significant improvement in tracking performance in the presence of occlusion. <s> BIB002 </s> A survey of datasets for visual tracking <s> Datasets for visual tracking <s> : Many real-world visual tracking applications have a high dimensionality, i.e. the system state is defined by a large number of variables. This kind of problem can be modelled as a dynamic optimization problem, which involves dynamic variables whose values change in time. Most applied research on optimization methods have focused on static optimization problems but these static methods often lack explicit adaptive methodologies. Heuristics are specific methods for solving problems in the absence of an algorithm for formal proof. Metaheuristics are approximate optimization methods which have been applied to more general problems with significant success. However, particle filters are Monte Carlo algorithms which solve the sequential estimation problem by approximating the theoretical distributions in the state space by simulated random measures called particles. However, particle filters lack efficient search strategies. In this paper, we propose a general framework to hybridize heuristics/metaheuristics with particle filters properly. The aim of this framework is to devise effective hybrid visual tracking algorithms naturally, guided by the use of abstraction techniques. Resulting algorithms exploit the benefits of both complementary approaches. As a particular example, a memetic algorithm particle filter is derived from the proposed hybridization framework. Finally, we show the performance of the memetic algorithm particle filter when it is applied to a multiple object tracking problem. <s> BIB003 </s> A survey of datasets for visual tracking <s> Datasets for visual tracking <s> Many researchers argue that fusing multiple cues increases the reliability and robustness of visual tracking. However, how the multi-cue integration is realized during tracking is still an open issue. In this work, we present a novel data fusion approach for multi-cue tracking using particle filter. Our method differs from previous approaches in a number of ways. First, we carry out the integration of cues both in making predictions about the target object and in verifying them through observations. Our second and more significant contribution is that both stages of integration directly depend on the dynamically changing reliabilities of visual cues. These two aspects of our method allow the tracker to easily adapt itself to the changes in the context, and accordingly improve the tracking accuracy by resolving the ambiguities. <s> BIB004 </s> A survey of datasets for visual tracking <s> Datasets for visual tracking <s> Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field. <s> BIB005
This section is dedicated to a selection of the datasets publicly available on the web that cover most of the problems in visual tracking (see Sect. 3 for details). We have chosen to describe these datasets by alphabetical order (note that we refer to some of them by their author's name-a personal choice). All the video sequences they contain are listed in Tables 1 and 2 (in chronological order of their appearance on the web), as well as some information about the number and the size of their frames, their color properties and where to find their ground truth, when available. These tables also mention the main issues for visual tracking raised by these sequences. We have classified these difficulties into nine categories: (1) illumination effects, (2) scene clutter, (3) appearance changes, (4) abrupt motion, (5) occlusions, (6) appearance/disappearance, (7) quality of frames, (8) similar appearance (this also means multiple objects can be tracked), (9) camera motion. For some of these categories, some additional information are provided. Concerning the appearance changes, for instance, we distinguish five cases: "scale" refers to zoom in or out cases, "view" to the changes in the point of view (corresponding to depth rotations), "shape" refers to deformable objects, "look" refers to variations such as glass removing, facial expressions, etc., and "rotation" to image plane rotations. The quality of the frames can be altered by Name of the sequence and of the dataset it belongs to, year, number of frames, resolution, color or not, ground truth availability, properties and difficulties for visual tracking [(1) illumination effects, (2) scene clutter, (3) appearance changes, (4) abrupt motion, (5) occlusions, (6) appearance/disappearance, (7) quality of frames, (8) similar appearance, (9) camera motion] and minimum tracking errors in the Tracker Benchmark BIB005 blur, fuzzy or block artifacts due to video compression. Note that video sequences from BEHAVE, CAVIAR and PETS do not appear in these tables because these datasets contain many video sequences: short descriptions of them are given respectively in Sects. 4.1, 4.6 and 4.17. We have also reported in these tables minimum tracking errors (in pixels) for sequences that were part of the Tracker Benchmark BIB005 , as well as the algorithms that provided these scores. This indicates which algorithms are the current best ones for these sequences. Name of the sequence and of the dataset it belongs to, year, number of frames, resolution, color or not, ground truth availability, properties and difficulties for visual tracking [(1) illumination effects, (2) scene clutter, (3) appearance changes, (4) abrupt motion, (5) occlusions, (6) appearance/disappearance, (7) quality of frames, (8) similar appearance, (9) camera motion] and minimum tracking errors in the Tracker Benchmark BIB005 4. BIB001 . A very complete ground truth visual tracking information is provided for all the sequences but one. It is composed of the bounding box for each interactive person, as well as labels in case of interactions, in XML format. Although this dataset was mainly proposed for behavior analysis of interacting groups, some of its sequences are used to validate visual tracking algorithms that consider occlusions or similar appearance objects  BIB004 BIB002 , or fast and varying motions of objects BIB003 for example. This dataset is used by a very large computer vision community working on video tracking, motion analysis and scene understanding.