Dynamic texture and scene classification are two fundamental problems in understanding natural video content. Extracting robust and effective features is a crucial step towards solving these problems. However the existing approaches suffer from the sensitivity to either varying illumination, or viewpoint changing, or even camera motion, and or the lack of spatial information. Inspired by the success of deep structures in image classification, we attempt to leverage a deep structure to extract feature for dynamic texture and scene classification. To tackle with the challenges in training a deep structure, we propose to transfer some prior knowledge from image domain to video domain. To be specific, we propose to apply a well-trained Convolutional Neural Network (ConvNet) as a mid-level feature extractor to extract features from each frame, and then form a representation of a video by concatenating the first and the second order statistics over the mid-level features. We term this two-level feature extraction scheme as a Transferred ConvNet Feature (TCoF). Moreover we explore two different implementations of the TCoF scheme, i.e., the TCoF and the TCoF, in which the mean-removed frames and the difference between two adjacent frames are used as the inputs of the ConvNet, respectively. We evaluate systematically the proposed spatial TCoF and the temporal TCoF schemes on three benchmark data sets, including DynTex, YUPENN, and Maryland, and demonstrate that the proposed approach yields superior performance.
The research history of dynamic texture classification is much longer than that of the dynamic scene. The later, as far as we know, started since two dynamic scene data sets -- Maryland Dynamic Scene data set in the wild'' @cite_54 and York stabilized Dynamic Scene data set @cite_8 -- were released. Although there might not be a clear distinction in nature, the slight difference of dynamic texture from dynamic scene is that the frames in a video of dynamic texture consist of images with richer texture whereas the frames in a video of dynamic scene are a natural scene involving over time. In addition, having mentioned of the data sets, compared to dynamic textures which are usually stabilized videos, the dynamic scene data set might include some significant camera motions.
@cite_54 @cite_8
1946072890 2951619115 2160144769 2006963646
The task of classifying videos of natural dynamic scenes into appropriate classes has gained lot of attention in recent years. The problem especially becomes challenging when the camera used to capture the video is dynamic. In this paper, we analyse the performance of statistical aggregation (SA) techniques on various pre-trained convolutional neural network(CNN) models to address this problem. The proposed approach works by extracting CNN activation features for a number of frames in a video and then uses an aggregation scheme in order to obtain a robust feature descriptor for the video. We show through results that the proposed approach performs better than the-state-of-the arts for the Maryland and YUPenn dataset. The final descriptor obtained is powerful enough to distinguish among dynamic scenes and is even capable of addressing the scenario where the camera motion is dominant and the scene dynamics are complex. Further, this paper shows an extensive study on the performance of various aggregation methods and their combinations. We compare the proposed approach with other dynamic scene classification algorithms on two publicly available datasets - Maryland and YUPenn to demonstrate the superior performance of the proposed approach. Dynamic texture and scene classification are two fundamental problems in understanding natural video content. Extracting robust and effective features is a crucial step towards solving these problems. However the existing approaches suffer from the sensitivity to either varying illumination, or viewpoint changing, or even camera motion, and or the lack of spatial information. Inspired by the success of deep structures in image classification, we attempt to leverage a deep structure to extract feature for dynamic texture and scene classification. To tackle with the challenges in training a deep structure, we propose to transfer some prior knowledge from image domain to video domain. To be specific, we propose to apply a well-trained Convolutional Neural Network (ConvNet) as a mid-level feature extractor to extract features from each frame, and then form a representation of a video by concatenating the first and the second order statistics over the mid-level features. We term this two-level feature extraction scheme as a Transferred ConvNet Feature (TCoF). Moreover we explore two different implementations of the TCoF scheme, i.e., the TCoF and the TCoF, in which the mean-removed frames and the difference between two adjacent frames are used as the inputs of the ConvNet, respectively. We evaluate systematically the proposed spatial TCoF and the temporal TCoF schemes on three benchmark data sets, including DynTex, YUPENN, and Maryland, and demonstrate that the proposed approach yields superior performance. The dynamic texture is a stochastic video model that treats the video as a sample from a linear dynamical system. The simple model has been shown to be surprisingly useful in domains such as video synthesis, video segmentation, and video classification. However, one major disadvantage of the dynamic texture is that it can only model video where the motion is smooth, i.e. video textures where the pixel values change smoothly. In this work, we propose an extension of the dynamic texture to address this issue. Instead of learning a linear observation function with PCA, we learn a non-linear observation function using kernel-PCA. The resulting kernel dynamic texture is capable of modeling a wider range of video motion, such as chaotic motion (e.g. turbulent water) or camera motion (e.g. panning). We derive the necessary steps to compute the Martin distance between kernel dynamic textures, and then validate the new model through classification experiments on video containing camera motion. The large-scale images and videos are one kind of the main source of big data. Dynamic texture (DT) is essential for understanding the video sequences with spatio-temporal similarities. This paper presents a powerful tool called dynamic fractal analysis to DT description and classification, which integrates rich description of DT with strong robustness to environmental changes. The proposed dynamic fractal spectrum (DFS) for DT sequences is composed of two components. The first one is a volumetric dynamic fractal spectrum component (V-DFS) that captures the stochastic self-similarities of DT sequences by treating them as 3D volumes; the second one is a multi-slice dynamic fractal spectrum component (S-DFS) that encodes fractal structures of repetitive DT patterns on 2D slices along different views of the 3D volume. To fully exploit various types of dynamic patterns in DT, five measurements of DT pixels are collected for the analysis on DT sequences from different perspectives. We evaluated our method on four publicly available benchmark datasets. All the experimental results have demonstrated the excellent performance of our method in comparison with state-of-the-art approaches. HighlightsThe dynamic multi-fractal analysis is developed for DT description.Our method is very discriminative and robust to environmental changes.A computational acceleration scheme is provided for the proposed descriptor.Our method exhibits excellent performance on four benchmark datasets.