Siamese Learning Visual Tracking: A Survey <s> Network Branches <s> Visual object tracking is a challenging computer vision problem with numerous real-world applications. This paper investigates the impact of convolutional features for the visual tracking problem. We propose to use activations from the convolutional layer of a CNN in discriminative correlation filter based tracking frameworks. These activations have several advantages compared to the standard deep features (fully connected layers). Firstly, they miti-gate the need of task specific fine-tuning. Secondly, they contain structural information crucial for the tracking problem. Lastly, these activations have low dimensionality. We perform comprehensive experiments on three benchmark datasets: OTB, ALOV300++ and the recently introduced VOT2015. Surprisingly, different to image classification, our results suggest that activations from the first layer provide superior tracking performance compared to the deeper layers. Our results further show that the convolutional features provide improved results compared to standard hand-crafted features. Finally, results comparable to state-of-the-art trackers are obtained on all three benchmark datasets. <s> BIB001 </s> Siamese Learning Visual Tracking: A Survey <s> Network Branches <s> In this paper we present a tracker, which is radically different from state-of-the-art trackers: we apply no model updating, no occlusion detection, no combination of trackers, no geometric matching, and still deliver state-of-theart tracking performance, as demonstrated on the popular online tracking benchmark (OTB) and six very challenging YouTube videos. The presented tracker simply matches the initial patch of the target in the first frame with candidates in a new frame and returns the most similar patch by a learned matching function. The strength of the matching function comes from being extensively trained generically, i.e., without any data of the target, using a Siamese deep neural network, which we design for tracking. Once learned, the matching function is used as is, without any adapting, to track previously unseen targets. It turns out that the learned matching function is so powerful that a simple tracker built upon it, coined Siamese INstance search Tracker, SINT, which only uses the original observation of the target from the first frame, suffices to reach state-of-theart performance. Further, we show the proposed tracker even allows for target re-identification after the target was absent for a complete video shot. <s> BIB002 </s> Siamese Learning Visual Tracking: A Survey <s> Network Branches <s> The Correlation Filter is an algorithm that trains a linear template to discriminate between images and their translations. It is well suited to object tracking because its formulation in the Fourier domain provides a fast solution, enabling the detector to be re-trained once per frame. Previous works that use the Correlation Filter, however, have adopted features that were either manually designed or trained for a different task. This work is the first to overcome this limitation by interpreting the Correlation Filter learner, which has a closed-form solution, as a differentiable layer in a deep neural network. This enables learning deep features that are tightly coupled to the Correlation Filter. Experiments illustrate that our method has the important practical benefit of allowing lightweight architectures to achieve state-of-the-art performance at high framerates. <s> BIB003 </s> Siamese Learning Visual Tracking: A Survey <s> Network Branches <s> The main challenges of visual object tracking arise from the arbitrary appearance of the objects that need to be tracked. Most existing algorithms try to solve this problem by training a new model to regenerate or classify each tracked object. As a result, the model needs to be initialized and retrained for each new object. In this paper, we propose to track different objects in an object-independent approach with a novel two-flow convolutional neural network (YCNN). The YCNN takes two inputs (one is an object image patch, the other is a larger searching image patch), then outputs a response map which predicts how likely and where the object would appear in the search patch. Unlike the object-specific approaches, the YCNN is actually trained to measure the similarity between the two image patches. Thus, this model will not be limited to any specific object. Furthermore, the network is end-to-end trained to extract both shallow and deep dedicated convolutional features for visual tracking. And once properly trained, the YCNN can be used to track all kinds of objects without further training and updating. As a result, our algorithm is able to run at a very high speed of 45 frames-per-second. The effectiveness of the proposed algorithm can also be proved by the experiments on two popular data sets: OTB-100 and VOT-2014. <s> BIB004
All proposed methods suggest convolutional branches inherited either from AlexNet or VGGNet with five conv layers except for YCNN that complements three layers by two fc layers and CFNet that studies one to five layers. The inheritance from AlexNet and VGGNet allows transfer learning from ImageNet and ALOV. The methods consider equal branches by effectively sharing the parameters which avoids during the fine-tuning overfitting to the small datasets currently available BIB002 . The first two conv layers capture very local visual detail, for example edges, contributing to the accuracy of the tracker, while conv layers three to five aggregate this detail to an object specific description, for example category specific details, which is important for the robustness of the tracker BIB001 . Max-pooling as it is part of AlexNet and VGGNet introduces invariance to deformations of the object but it also reduces significantly image resolution and by that hinders improvements to the tracker's accuracy. All authors except BIB004 recognise this limitation and use two max-pooling layers to trade-off accuracy and deformation invariance. BIB003 also show important insights into the number of layers. They report saturation of tracking performance with increasing network depth and that more than five conv layers yield minor performance gains. CFNet implements the object specific description on higher layers with a correlation filter which allows an effective object specific description and fast computation in the Fourier domain. This significantly improves computational performance and shows that the branches are representable in various ways by combining layers of heterogeneous features.