Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> III. OBJECT TRACKING <s> We explore an original strategy for building deep networks, based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs. The resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders. It is however shown on a benchmark of classification problems to yield significantly lower classification error, thus bridging the performance gap with deep belief networks (DBN), and in several cases surpassing it. Higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent SVM classifiers. Qualitative experiments show that, contrary to ordinary autoencoders, denoising autoencoders are able to learn Gabor-like edge detectors from natural image patches and larger stroke detectors from digit images. This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations. <s> BIB001 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> III. OBJECT TRACKING <s> We apply salient feature detection and tracking in videos to simulate fixations and smooth pursuit in human vision. With tracked sequences as input, a hierarchical network of modules learns invariant features using a temporal slowness constraint. The network encodes invariance which are increasingly complex with hierarchy. Although learned from videos, our features are spatial instead of spatial-temporal, and well suited for extracting features from still images. We applied our features to four datasets (COIL-100, Caltech 101, STL-10, PubFig), and observe a consistent improvement of 4% to 5% in classification accuracy. With this approach, we achieve state-of-the-art recognition accuracy 61% on STL-10 dataset. <s> BIB002 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> III. OBJECT TRACKING <s> In this paper, we study the challenging problem of tracking the trajectory of a moving object in a video with possibly very complex background. In contrast to most existing trackers which only learn the appearance of the tracked object online, we take a different approach, inspired by recent advances in deep learning architectures, by putting more emphasis on the (unsupervised) feature learning problem. Specifically, by using auxiliary natural images, we train a stacked de-noising autoencoder offline to learn generic image features that are more robust against variations. This is then followed by knowledge transfer from offline training to the online tracking process. Online tracking involves a classification neural network which is constructed from the encoder part of the trained autoencoder as a feature extractor and an additional classification layer. Both the feature extractor and the classifier can be further tuned to adapt to appearance changes of the moving object. Comparison with the state-of-the-art trackers on some challenging benchmark video sequences shows that our deep learning tracker is more accurate while maintaining low computational cost with real-time performance when our MATLAB implementation of the tracker is used with a modest graphics processing unit (GPU). <s> BIB003 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> III. OBJECT TRACKING <s> Defining hand-crafted feature representations needs expert knowledge, requires timeconsuming manual adjustments, and besides, it is arguably one of the limiting factors of object tracking. In this paper, we propose a novel solution to automatically relearn the most useful feature representations during the tracking process in order to accurately adapt appearance changes, pose and scale variations while preventing from drift and tracking failures. We employ a candidate pool of multiple Convolutional Neural Networks (CNNs) as a data-driven model of different instances of the target object. Individually, each CNN maintains a specific set of kernels that favourably discriminate object patches from their surrounding background using all available low-level cues. These kernels are updated in an online manner at each frame after being trained with just one instance at the initialization of the corresponding CNN. Given a frame, the most promising CNNs in the pool are selected to evaluate the hypothesises for the target object. The hypothesis with the highest score is assigned as the current detection window and the selected models are retrained using a warm-start back-propagation which optimizes a structural loss function. In addition to the model-free tracker, we introduce a class-specific version of the proposed method that is tailored for tracking of a particular object class such as human faces. Our experiments on a large selection of videos from the recent benchmarks demonstrate that our method outperforms the existing state-of-the-art algorithms and rarely loses the track of the target object. <s> BIB004 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> III. OBJECT TRACKING <s> In this paper, we propose an approach to learn hierarchical features for visual object tracking. First, we offline learn features robust to diverse motion patterns from auxiliary video sequences. The hierarchical features are learned via a two-layer convolutional neural network. Embedding the temporal slowness constraint in the stacked architecture makes the learned features robust to complicated motion transformations, which is important for visual object tracking. Then, given a target video sequence, we propose a domain adaptation module to online adapt the pre-learned features according to the specific target object. The adaptation is conducted in both layers of the deep feature learning module so as to include appearance information of the specific target object. As a result, the learned hierarchical features can be robust to both complicated motion transformations and appearance changes of target objects. We integrate our feature learning algorithm into three tracking methods. Experimental results demonstrate that significant improvement can be achieved using our learned hierarchical features, especially on video sequences with complicated motion transformations. <s> BIB005
Object tracking is intended to locate a target object in a video sequence given its location in the first frame. Recently, some deep learning based tracking algorithms have achieved very promising results. We review some representative works as follows. Wang and Yeung BIB003 propose to learn deep and compact features for visual tracking by using stacked denoising autoencoder BIB001 . The network architecture is illustrated in Figure 6 . It is reported that using the learned deep features can outperform other 7 state-of-the-art trackers on 10 sequences in terms of two tracking measurements: average center error (7.3 pixels) and average success rate (85.5%). Additionally, the proposed tracker with deep features achieved an average frame rate of 15fps which is suitable for real-time applications. Li et al. BIB004 propose to learn discriminative feature representations for visual tracking by using convolutional neural networks. An overview of the method is illustrated in Figure 7 . It can be observed that a pool of multiple convolutional neural networks are utilized to maintain different kernels regarding all possible low-level cues, which aim to discriminate object patches from their surrounding background. Given a frame, the most prospective convolutional neural network in the pool is used to predict the new location of the target object. Meanwhile, the selected network is retrained using a warmstart backpropagation scheme. Also, we can observe from Figure 7 that a class-specific convolutional neural network is involved in the whole architecture, which is helpful for tracking a certain class of objects (e.g. a person's face). Wang et al. BIB005 propose to learn hierarchical features robust to both complicated motion transformations and target appearance changes. The stacked architecture and the adap- Fig. 8 . Illustration of the stacked architecture and the adaptation module proposed by Wang et al. BIB005 . tation module are illustrated in Figure 8 . First, the generic features robust to complicated motion transformations are offline learned from auxiliary video data by using a twolayer neural networks under the temporal constraints BIB002 . Then, the pre-learned features are online adapted according to the specific target object sequence. As a result, the adapted features are able to capture appearance changes of target objects. For example, the proposed tracker can handle not only non-rigid deformation of a basketball player's body but also the specific appearance changes. It is reported that using the feature learning algorithm can significantly improve tracking performance, especially on the sequences with complex motion transformations. Object tracking can be applied to surveillance systems of smart cities. It is important to automatically track suspected people or target vehicles for safety monitoring and urban flow management. Deep learning can leverage smart city big data to train deep models which are more robust to visual variations of target objects than traditional models. Therefore, smart city tracking systems can be enhanced by using deep learning algorithms for handling large amount of video data.