Siamese Learning Visual Tracking: A Survey <s> Tracker Design <s> Massachusetts Institute of Technology. Dept. of Electrical Engineering. Thesis. 1970. Ph.D. <s> BIB001 </s> Siamese Learning Visual Tracking: A Survey <s> Tracker Design <s> Abstract How are our brains functionally organized to achieve adaptive behavior in a changing world? This article presents one alternative to the computer analogy that suggests brains are organized into independent modules. Evidence is reviewed that brains are in fact organized into parallel processing streams with complementary properties. Hierarchical interactions within each stream and parallel interactions between streams create coherent behavioral representations that overcome the complementary deficiencies of each stream and support unitary conscious experiences. This perspective suggests how brain design reflects the organization of the physical world with which brains interact. Examples from perception, learning, cognition and action are described, and theoretical concepts and mechanisms by which complementarity might be accomplished are presented. <s> BIB002 </s> Siamese Learning Visual Tracking: A Survey <s> Tracker Design <s> We propose a novel attentional model for simultaneous object tracking and recognition that is driven by gaze data. Motivated by theories of the human perceptual system, the model consists of two interacting pathways: ventral and dorsal. The ventral pathway models object appearance and classification using deep (factored)-restricted Boltzmann machines. At each point in time, the observations consist of retinal images, with decaying resolution toward the periphery of the gaze. The dorsal pathway models the location, orientation, scale and speed of the attended object. The posterior distribution of these states is estimated with particle filtering. Deeper in the dorsal pathway, we encounter an attentional mechanism that learns to control gazes so as to minimize tracking uncertainty. The approach is modular (with each module easily replaceable with more sophisticated algorithms), straightforward to implement, practically efficient, and works well in simple video sequences. <s> BIB003 </s> Siamese Learning Visual Tracking: A Survey <s> Tracker Design <s> Several benchmark datasets for visual tracking research have been created in recent years. Despite their usefulness, whether they are sufficient for understanding and diagnosing the strengths and weaknesses of different trackers remains questionable. To address this issue, we propose a framework by breaking a tracker down into five constituent parts, namely, motion model, feature extractor, observation model, model updater, and ensemble post-processor. We then conduct ablative experiments on each component to study how it affects the overall result. Surprisingly, our findings are discrepant with some common beliefs in the visual tracking research community. We find that the feature extractor plays the most important role in a tracker. On the other hand, although the observation model is the focus of many studies, we find that it often brings no significant improvement. Moreover, the motion model and model updater contain many details that could affect the result. Also, the ensemble post-processor can improve the result substantially when the constituent trackers have high diversity. Based on our findings, we put together some very elementary building blocks to give a basic tracker which is competitive in performance to the state-of-the-art trackers. We believe our framework can provide a solid baseline when conducting controlled experiments for visual tracking research. <s> BIB004
The paradigm shift from pure hand-crafted designs to connectionism in computer vision suggests to design trackers as multiple complementary and loosely coupled learning processes which are finally embedded and intertwined into some kind of management and control process BIB001 ,  . There is evidence in neuroscience and cognitive science that the visual brain and mind works in similar way BIB002 ,  . The motivation comes from the outstanding success of convolutional networks as solution to categorial object recognition and by the higher complexity of tracking compared to categorial object recognition. 2. Other names are (hidden) state variables or variables. Each learning process is a hierarchical meaningful description of the visual input, with the aim to yield evidence of the tracker's state and to finally make predictions of the particular latent variables given by the design. The tracker under this paradigm works basically in two operating states either it is learning from incoming visual input to adapt the parameters of the learning processes by end-to-end training or the tracker is inferring the latent variables of the tracker. The data flow of such a tracker's design shows three fundamental pathways BIB003 which Fig. 1 Trackers are inference machines, gathering evidence of who and where by collecting sequentially observations. Parametric descriptions are precisely formulated and initially given. Each prediction step completes the inference process by computing the latent variables given the evidence of who and where. The third why pathway formalises the ob-jective function so that the tracker can come to a conclusion. For example, a popular objective is least-squares to given ground truth, e.g. a given image template of the object. Different interconnected modules capturing the various descriptions of the tracking problem constitute these pathways. The complexity of interconnection and the choice of the mathematical formulation has direct consequence to the robustness and accuracy of the tracking results as well as to the computational efficiency of the tracking algorithm. Popular designs are mode seeking, HMM's, Kalman filter, variants and generalised SMC methods. Recently BIB004 proposed five key building blocks of a tracker: motion model, feature extractor, observation model, model updater, ensemble postprocessor.