Siamese Learning Visual Tracking: A Survey <s> Connection of Branches <s> Machine learning techniques are often used in computer vision due to their ability to leverage large amounts of training data to improve performance. Unfortunately, most generic object trackers are still trained from scratch online and do not benefit from the large number of videos that are readily available for offline training. We propose a method for offline training of neural networks that can track novel objects at test-time at 100 fps. Our tracker is significantly faster than previous methods that use neural networks for tracking, which are typically very slow to run and not practical for real-time applications. Our tracker uses a simple feed-forward network with no online training required. The tracker learns a generic relationship between object motion and appearance and can be used to track novel objects that do not appear in the training set. We test our network on a standard tracking benchmark to demonstrate our tracker's state-of-the-art performance. Further, our performance improves as we add more videos to our offline training set. To the best of our knowledge, our tracker is the first neural-network tracker that learns to track generic objects at 100 fps. <s> BIB001 </s> Siamese Learning Visual Tracking: A Survey <s> Connection of Branches <s> The main challenges of visual object tracking arise from the arbitrary appearance of the objects that need to be tracked. Most existing algorithms try to solve this problem by training a new model to regenerate or classify each tracked object. As a result, the model needs to be initialized and retrained for each new object. In this paper, we propose to track different objects in an object-independent approach with a novel two-flow convolutional neural network (YCNN). The YCNN takes two inputs (one is an object image patch, the other is a larger searching image patch), then outputs a response map which predicts how likely and where the object would appear in the search patch. Unlike the object-specific approaches, the YCNN is actually trained to measure the similarity between the two image patches. Thus, this model will not be limited to any specific object. Furthermore, the network is end-to-end trained to extract both shallow and deep dedicated convolutional features for visual tracking. And once properly trained, the YCNN can be used to track all kinds of objects without further training and updating. As a result, our algorithm is able to run at a very high speed of 45 frames-per-second. The effectiveness of the proposed algorithm can also be proved by the experiments on two popular data sets: OTB-100 and VOT-2014. <s> BIB002
All proposed methods except SINT connect the branches, SiamFC and CFNet with a single crosscorrelation layer, YCNN and GOTURN with three fc layers. SINT omits the concatenating layer by using normalisation layers at the end of both branches. Fig. 2 illustrates these three variations of network architecture. This Siamese network architecture of SiamFC, CFNet and SINT in combination with parameter sharing limit the feature selection to the spatial image domain. Instead, YCNN and GO-TURN enable additional learning of spatiotemporal features in the concatenating layers, as argued by BIB001 such as "relationships between an object's appearance change and its motion" which seems very general for different categories of objects. Parameter sharing has the consequence that all methods require appearance constancy between template and search region, hence BIB002 's argument that YCNN's deep features show "superiority of recognising an object with varying appearance" is questionable. SiamFC, Theoretically, the network of GOTURN generalises over the network of SiamFC which allows capturing features beyond sole visual features of the exemplar image and which allows regression of the bounding box instead of convoluting a final score map capturing potential positions of the exemplar image within the search image. The author's argue that GOTURN learns a generic relationship between arbitrary motion and visual features, however this is not clear yet. Due to the more general Y-shaped architecture it might learn features beyond pure visual such as motion and their relationship in the fully-connected layers, however the network might also be able to learn context features as well. CFNet and SINT assume a specific function of similarity and the idea is to solely learn visual features to best match the given similarity. SINT even expresses similarity by the training loss which might have advantages in generalisation as particular different functions of similarity and training loss might derive adversary optimisation problems. This is not a problem for YCNN and GOTURN, as similarity and features are jointly learned, however, the interference with the particular training loss is unclear.