A survey of datasets for visual tracking <s> Discussion <s> We propose a novel tracking algorithm that can work robustly in a challenging scenario such that several kinds of appearance and motion changes of an object occur at the same time. Our algorithm is based on a visual tracking decomposition scheme for the efficient design of observation and motion models as well as trackers. In our scheme, the observation model is decomposed into multiple basic observation models that are constructed by sparse principal component analysis (SPCA) of a set of feature templates. Each basic observation model covers a specific appearance of the object. The motion model is also represented by the combination of multiple basic motion models, each of which covers a different type of motion. Then the multiple basic trackers are designed by associating the basic observation models and the basic motion models, so that each specific tracker takes charge of a certain change in the object. All basic trackers are then integrated into one compound tracker through an interactive Markov Chain Monte Carlo (IMCMC) framework in which the basic trackers communicate with one another interactively while run in parallel. By exchanging information with others, each tracker further improves its performance, which results in increasing the whole performance of tracking. Experimental results show that our method tracks the object accurately and reliably in realistic videos where the appearance and motion are drastically changing over time. <s> BIB001 </s> A survey of datasets for visual tracking <s> Discussion <s> In this paper we propose a robust object tracking algorithm using a collaborative model. As the main challenge for object tracking is to account for drastic appearance change, we propose a robust appearance model that exploits both holistic templates and local representations. We develop a sparsity-based discriminative classifier (SD-C) and a sparsity-based generative model (SGM). In the S-DC module, we introduce an effective method to compute the confidence value that assigns more weights to the foreground than the background. In the SGM module, we propose a novel histogram-based method that takes the spatial information of each patch into consideration with an occlusion handing scheme. Furthermore, the update scheme considers both the latest observations and the original template, thereby enabling the tracker to deal with appearance change effectively and alleviate the drift problem. Numerous experiments on various challenging videos demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms. <s> BIB002 </s> A survey of datasets for visual tracking <s> Discussion <s> Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field. <s> BIB003 </s> A survey of datasets for visual tracking <s> Discussion <s> Adaptive tracking-by-detection methods are widely used in computer vision for tracking arbitrary objects. Current approaches treat the tracking problem as a classification task and use online learning techniques to update the object model. However, for these updates to happen one needs to convert the estimated object position into a set of labelled training examples, and it is not clear how best to perform this intermediate step. Furthermore, the objective for the classifier (label prediction) is not explicitly coupled to the objective for the tracker (estimation of object position). In this paper, we present a framework for adaptive visual object tracking based on structured output prediction. By explicitly allowing the output space to express the needs of the tracker, we avoid the need for an intermediate classification step. Our method uses a kernelised structured output support vector machine (SVM), which is learned online to provide adaptive tracking. To allow our tracker to run at high frame rates, we (a) introduce a budgeting mechanism that prevents the unbounded growth in the number of support vectors that would otherwise occur during tracking, and (b) show how to implement tracking on the GPU. Experimentally, we show that our algorithm is able to outperform state-of-the-art trackers on various benchmark videos. Additionally, we show that we can easily incorporate additional features and kernels into our framework, which results in increased tracking performance. <s> BIB004
We have listed in the preceding subsections numerous visual tracking datasets and given, for each one, its features (which visual tracking issues were raised). We have also described how these datasets are annotated. The other datasets involved a high level of difficulties for visual tracking, ranging from highly deforming objects [20, 25,   46, 47] , abrupt motions [20, 23, 24] , severe occlu- BIB004 sions  objects entering or living the field of view [23] , to camera motions  20, 23, 36, 46] . The higher the number of issues a visual tracker will be able to deal with, the more it will be fit for the new perspectives offered by the current challenges in visual tracking. Table 3 summarizes the results obtained in the Tracker Benchmark BIB003 that compared 29 visual tracking algorithms. Here we can find, for each of the visual tracking difficulties, the average, highest and lowest tracking errors (in pixels) obtained in the benchmark. We also report the algorithm that achieved the best score in this benchmark. This table shows that the best current visual tracking algorithms are SCM BIB002 , VTD BIB001 , TLD  and STRUCK BIB004 . Note that the tested video sequences are a subset of those presented in this article. The lowest tracking errors for this subset of sequences are reported in Tables 1 and 2 . With the development of RGB-D sensors, lots of video datasets have been proposed to test visual tracking including depth information. Some of them are cited in Sect. 5 depending on their objective. Even if this does not exactly refer to visual tracking, including depth information into a visual tracker enables tracking in three dimensions as well as analyzing movements with more precision. Another difficulty for visual tracking is when motion comes from both the tracked object and the camera that is acquiring the scene. Some datasets with this feature have already been proposed; for example, for traffic surveillance when the camera is positioned in a car windscreen driving on a road [110] [111] [112] . In the same spirit, developing first-person or egocentric view datasets  is required because of a growing field of research due to the miniaturization of cameras and to the development of visual sensors on body (Google glasses, GoPro camera, etc.). Aerial images are also challenging for visual tracking tasks because all the objects to track have similar appearances  115] . Finally, a visual tracker can also be tested on datasets that were acquired in specific modalities such as infrared  or live cell imaging  . Tracking during long time periods is still a challenging problem. It could be very useful for the community to provide new data that include a unique issue (illumination changes, single appearance changes-shape, or scale, or view, or look-, motion directions, clutter, etc.) observable during a long time. This would allow testing the capacity of visual trackers not to diverge with time by considering this specific issue. A major challenge for visual tracking datasets is to provide a correct GT, i.e., one that is constructed with high precision: a manual annotation seems to be, today, the best way to achieve such a precision but this is very tedious. To our opinion, the construction of a GT for visual tracking is still an open problem. Nevertheless, the community has developed its own solutions that are more often than not sufficient, even if they do not always consider all the visual tracking difficulties (which GT could be well adapted in case of blur, scale changes, appearance/disappearance problems, for example?). Of course, the comparison between the visual tracker's output and the GT is more relevant if both represent the same thing. For example, it can be hard to measure the quality of a visual tracker when its output is a rectangular region surrounding the tracked object whereas the GT's annotation corresponds to a group of pixels (a silhouette, a blob, etc.). As mentioned in the beginning of this section, we consider that a visual tracker's output is a shape surrounding the tracked object and that correctly estimating this shape is the purpose of the visual tracker. However, today, for many applications, this is not the ultimate goal but rather just a mean to achieve a more complex goal, e.g., analyzing the behaviors of individuals in a scene. Therefore, in addition to the aforementioned visual tracking difficulties, these applications raise new challenges and, thus, require other datasets to enable researchers to assess the effectiveness of their algorithms. The next section describes some of these datasets.