A survey of datasets for visual tracking <s> Introduction <s> This report contains a review of visual tracking in monocular video sequences. For the purpose of this review, the majority of the visual trackers in the literature are divided into three tracking categories: discrete feature trackers, contour trackers, and region-based trackers. This categorization was performed based on the features used and the algorithms employed by the various visual trackers. The first class of trackers represents targets as discrete features (e.g. points, sets of points, lines) and performs data association using a distance metric that accommodates the particular feature. Contour trackers provide precise outlines of the target boundaries, meaning that they must not only uncover the position of the target, but its shape as well. Contour trackers often make use of gradient edge information during the tracking process. Region trackers represent the target with area-based descriptors that define its support and attempt to locate the image region in the current frame that best matches an object template. Trackers that are not in agreement with the abovementioned categorization, including those that combine methods from the three defined classes, are also considered in this review. In addition to categorizing and describing the various visual trackers in the literature, this review also provides a commentary on the current state of the field as well as a comparative analysis of the various approaches. The paper concludes with an outline of open problems in visual tracking. <s> BIB001 </s> A survey of datasets for visual tracking <s> Introduction <s> Although there is much research on behaviour recognition in time-varying video, there are few ground truthed datasets for assessing multi-person behavioral interactions. This short paper presents the BEHAVE project’s dataset, which has around 90,000 frames of humans identified by bounding boxes, with interacting groups classified into one of 5 different behaviors. An example of its use is also presented. <s> BIB002 </s> A survey of datasets for visual tracking <s> Introduction <s> Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field. <s> BIB003
"Visual tracking refers in general to the ability of the eyes to quickly and accurately look (fixate), visually follow a moving object (pursuit) and efficiently move our eyes so we can fixate on objects from point to point as in reading (saccades)" (Doctor Dan L. Fortenbacher, specialized in vision therapy). From the computer vision community point of view, visual tracking is the process of locating, identifying, and determining the dynamic configuration of one or many moving (possibly deformable) objects (or parts of objects) in each frame of one or several cameras BIB001  . As such, this involves solving many different challenging problems. What makes a good visual tracking algorithm is its capacity to handle all the variability in a video sequence caused by the tracked object, the scene and the camera acquiring the scene. Such variability can be caused, for instance, by pose and illumination variations, occlusions, varying and erratic motions. The algorithm should also be robust to cluttered backgrounds, blur, poor video acquisition quality and to the similarity between multiple objects to track. The combinations of these issues make visual tracking very challenging, and more and more emerging applications, such as video surveillance, require the algorithms to work under such very difficult conditions. This has led researchers to develop many visual tracking algorithms that can tackle, at least partially, some of these issues. They all have their own strengths and weaknesses and, to compare them, some datasets are regularly made publicly available (Fig. 1) . The goal of this paper is not to perform a review of all existing visual tracking algorithms (see BIB001  for two points of view on this matter, and  [4] BIB003 for benchmarks). This would be impossible anyway: indeed, too many works have been proposed and are still proposed every year since the mid- BIB001 The BEHAVE dataset BIB002 . Two snapshots of sequences, with the ground truth bounding boxes of the objects to track dle of the 90's. Rather, in this paper, we are interested in the way to evaluate and compare visual tracking algorithms. Our study is mainly articulated around an original cartography of visual tracking datasets depending on the issues involved in their video sequences. To our knowledge, this is the first study from this point of view. Of course, there exist websites providing some dataset repositories  but, more often than not, they do not describe the issues they address and, individually, each one only provides a small subset of all the datasets available. Therefore, they do not completely answer the above question. We hope that our cartography will be useful in that it gathers in a principled way many datasets scattered over the web. As such, we expect it should help researchers selecting easily the most appropriate datasets they need in order to evaluate their tracker's performances on specific visual tracking problems. As a side effect, this cartography should also prove useful for the evaluation of the components involved in more complex tasks than just visual tracking, e.g., human behavior or dynamic scene analysis. This article is organized as follows. Section 2 first defines the notion of visual tracking as a specific task and a unique goal, by opposition to it being only a component of a more complex framework where visual tracking is not a goal but a mean (e.g., it is only a subproblem involved in scene analysis problems). This distinction enables to discriminate existing datasets that are only devoted to visual tracking, i.e., those which are designed explicitly as challenges to visual tracking algorithms, from datasets whose challenges are not only related to visual tracking. Section 3 describes the different issues that naturally arise in visual tracking and the way a visual tracking algorithm can be evaluated. Section 4 presents datasets that are most useful to test specific properties of visual tracking algorithms. In particular, some of the algorithms that were tested on these datasets are given, as well as the visual tracking issues involved in the datasets. Section 5 is dedicated to a classification of datasets used for research problems in which visual tracking is a mean and not a goal. Here, two main classes of such problems are considered: human motion understanding and scene dynamics understanding. Finally, concluding remarks and perspectives are given in Sect. 6.