Dynamic textures exist in various forms, e.g., fire, smoke, and traffic jams, but recognizing dynamic texture is challenging due to the complex temporal variations. In this paper, we present a novel approach stemmed from slow feature analysis (SFA) for dynamic texture recognition. SFA extracts slowly varying features from fast varying signals. Fortunately, SFA is capable to leach invariant representations from dynamic textures. However, complex temporal variations require high-level semantic representations to fully achieve temporal slowness, and thus it is impractical to learn a high-level representation from dynamic textures directly by SFA. In order to learn a robust low-level feature to resolve the complexity of dynamic textures, we propose manifold regularized SFA (MR-SFA) by exploring the neighbor relationship of the initial state of each temporal transition and retaining the locality of their variations. Therefore, the learned features are not only slowly varying, but also partly predictable. MR-SFA for dynamic texture recognition is proposed in the following steps: 1) learning feature extraction functions as convolution filters by MR-SFA, 2) extracting local features by convolution and pooling, and 3) employing Fisher vectors to form a video-level representation for classification. Experimental results on dynamic texture and dynamic scene recognition datasets validate the effectiveness of the proposed approach.
High-level features have also been exploited for dynamic texture recognition. Deep learning has been successfully applied to general object recognition and detection. It has also been applied to dynamic texture recognition. A 3D convolutional neural network (CNN) was trained from a very large number of videos @cite_25 . This 3D CNN has been used as general video feature extractor, and achieved a good result on dynamic scene recognition. Many approaches use a pre-trained CNN as a high-level feature extractor @cite_16 @cite_32 @cite_8 . These approaches outperform most existing dynamic texture recognition approaches. Besides the CNN, a complex network was proposed to extract features from dynamic textures directly @cite_21 . A deep belief network was used to extract features from conventional features @cite_29 . In contrast to all of the above-mentioned approaches that are based on deeply learned networks, MR-SFA extracts features without using deep networks.
@cite_8 @cite_29 @cite_21 @cite_32 @cite_16 @cite_25
2950366620 2002128052 2042248823 1946072890 2951619115 2952633803
State-of-the-art image-set matching techniques typically implicitly model each image-set with a Gaussian distribution. Here, we propose to go beyond these representations and model image-sets as probability distribution functions (PDFs) using kernel density estimators. To compare and match image-sets, we exploit Csiszar f-divergences, which bear strong connections to the geodesic distance defined on the space of PDFs, i.e., the statistical manifold. Furthermore, we introduce valid positive definite kernels on the statistical manifolds, which let us make use of more powerful classification schemes to match image-sets. Finally, we introduce a supervised dimensionality reduction technique that learns a latent space where f-divergences reflect the class labels of the data. Our experiments on diverse problems, such as video-based face recognition and dynamic texture classification, evidence the benefits of our approach over the state-of-the-art image-set matching methods. In this paper, a novel framework is proposed for dynamic textures (DTs) recognition by learning a high level feature using deep neural network (DNN). The insight behind the method is that a DT appearing in different videos should share similar features, which can be learned for better recognition performance. Unlike many prior works only focus on low level or middle level features, we propose a novel high level feature learning method using DNN. Our goal is to construct a compact and discriminative semantic feature. The conventional bag of features approach using k-means is not semantically meaningful since the clustering criterion is based on appearance similarity. The proposed framework can effectively overcome the problem by capturing the semantic relations of the middle level by DNN. Extensive experiments with qualitative and quantitative results demonstrate the efficacy of our approach. Abstract In this paper, we propose a novel approach for dynamic texture representation based on complex networks. In the proposed approach, each pixel of the video is mapped into a node of the complex network. Initially, a regular complex network is obtained by connecting two nodes if the Euclidean distance between their related pixels is equal or less than a given radius. For each connection, a weight is defined by the difference of the pixel intensities. Given the regular complex network, a function is applied to remove connections whose weight is equal to or below a given threshold. Finally, a feature vector is obtained by calculating the spatial and temporal average degree for networks transformed by different values of threshold and radius. The number of connections of pixels from the same frame and from different frames, respectively, gives the spatial and temporal degrees. Experimental results using synthetic and real dynamic textures have demonstrated the effectiveness of the proposed approach. The task of classifying videos of natural dynamic scenes into appropriate classes has gained lot of attention in recent years. The problem especially becomes challenging when the camera used to capture the video is dynamic. In this paper, we analyse the performance of statistical aggregation (SA) techniques on various pre-trained convolutional neural network(CNN) models to address this problem. The proposed approach works by extracting CNN activation features for a number of frames in a video and then uses an aggregation scheme in order to obtain a robust feature descriptor for the video. We show through results that the proposed approach performs better than the-state-of-the arts for the Maryland and YUPenn dataset. The final descriptor obtained is powerful enough to distinguish among dynamic scenes and is even capable of addressing the scenario where the camera motion is dominant and the scene dynamics are complex. Further, this paper shows an extensive study on the performance of various aggregation methods and their combinations. We compare the proposed approach with other dynamic scene classification algorithms on two publicly available datasets - Maryland and YUPenn to demonstrate the superior performance of the proposed approach. Dynamic texture and scene classification are two fundamental problems in understanding natural video content. Extracting robust and effective features is a crucial step towards solving these problems. However the existing approaches suffer from the sensitivity to either varying illumination, or viewpoint changing, or even camera motion, and or the lack of spatial information. Inspired by the success of deep structures in image classification, we attempt to leverage a deep structure to extract feature for dynamic texture and scene classification. To tackle with the challenges in training a deep structure, we propose to transfer some prior knowledge from image domain to video domain. To be specific, we propose to apply a well-trained Convolutional Neural Network (ConvNet) as a mid-level feature extractor to extract features from each frame, and then form a representation of a video by concatenating the first and the second order statistics over the mid-level features. We term this two-level feature extraction scheme as a Transferred ConvNet Feature (TCoF). Moreover we explore two different implementations of the TCoF scheme, i.e., the TCoF and the TCoF, in which the mean-removed frames and the difference between two adjacent frames are used as the inputs of the ConvNet, respectively. We evaluate systematically the proposed spatial TCoF and the temporal TCoF schemes on three benchmark data sets, including DynTex, YUPENN, and Maryland, and demonstrate that the proposed approach yields superior performance. We propose a simple, yet effective approach for spatiotemporal feature learning using deep 3-dimensional convolutional networks (3D ConvNets) trained on a large scale supervised video dataset. Our findings are three-fold: 1) 3D ConvNets are more suitable for spatiotemporal feature learning compared to 2D ConvNets; 2) A homogeneous architecture with small 3x3x3 convolution kernels in all layers is among the best performing architectures for 3D ConvNets; and 3) Our learned features, namely C3D (Convolutional 3D), with a simple linear classifier outperform state-of-the-art methods on 4 different benchmarks and are comparable with current best methods on the other 2 benchmarks. In addition, the features are compact: achieving 52.8 accuracy on UCF101 dataset with only 10 dimensions and also very efficient to compute due to the fast inference of ConvNets. Finally, they are conceptually very simple and easy to train and use.