Representing videos by densely extracted local space-time features has recently become a popular approach for analysing actions. In this study, the authors tackle the problem of categorising human actions by devising bag of words (BoWs) models based on covariance matrices of spatiotemporal features, with the features formed from histograms of optical flow. Since covariance matrices form a special type of Riemannian manifold, the space of symmetric positive definite (SPD) matrices, non-Euclidean geometry should be taken into account while discriminating between covariance matrices. To this end, the authors propose to embed SPD manifolds to Euclidean spaces via a diffeomorphism and extend the BoW approach to its Riemannian version. The proposed BoW approach takes into account the manifold geometry of SPD matrices during the generation of the codebook and histograms. Experiments on challenging human action datasets show that the proposed method obtains notable improvements in discrimination accuracy, in comparison with several state-of-the-art methods.
An alternative line of research proposes to track given spatial point over time and capture related information. Messing al @cite_37 track Harris3D @cite_23 interest points with a KLT tracker @cite_26 and extract velocity history information. To improve performance, other useful features such as appearance and location are taken into account in a generative mixture model. Recently, Wang al @cite_42 show promising results by tracking densely sampled points and extract aligned shape, appearance, and motion features. They also introduce Motion Boundary Histograms (MBH) based on differential optical flow.
@cite_37 @cite_26 @cite_42 @cite_23
1530781137 2110558047 2112329933 2016802777
Dense and accurate motion tracking is an important requirement for many video feature extraction algorithms. In this paper we provide a method for computing point trajectories based on a fast parallel implementation of a recent optical flow algorithm that tolerates fast motion. The parallel implementation of large displacement optical flow runs about 78× faster than the serial C++ version. This makes it practical to use in a variety of applications, among them point tracking. In the course of obtaining the fast implementation, we also proved that the fixed point matrix obtained in the optical flow technique is positive semi-definite. We compare the point tracking to the most commonly used motion tracker - the KLT tracker - on a number of sequences with ground truth motion. Our resulting technique tracks up to three orders of magnitude more points and is 46 more accurate than the KLT tracker. It also provides a tracking density of 48 and has an occlusion error of 3 compared to a density of 0.1 and occlusion error of 8 for the KLT tracker. Compared to the Particle Video tracker, we achieve 66 better accuracy while retaining the ability to handle large displacements while running an order of magnitude faster. We present an integrated probabilistic model for layered object tracking that combines dynamics on implicit shape representations, topological shape constraints, adaptive appearance models, and layered flow. The generative model combines the evolution of appearances and layer shapes with a Gaussian process flow and explicit layer ordering. Efficient MCMC sampling algorithms are developed to enable a particle filtering approach while reasoning about the distribution of object boundaries in video. We demonstrate the utility of the proposed tracking algorithm on a wide variety of video sources while achieving state-of-the-art results on a boundary-accurate tracking dataset. Evaluation of tracking algorithms in the absence of ground truth is a challenging problem. There exist a variety of approaches for this problem, ranging from formal model validation techniques to heuristics that look for mismatches between track properties and the observed data. However, few of these methods scale up to the task of visual tracking, where the models are usually nonlinear and complex and typically lie in a high-dimensional space. Further, scenarios that cause track failures and or poor tracking performance are also quite diverse for the visual tracking problem. In this paper, we propose an online performance evaluation strategy for tracking systems based on particle filters using a time-reversed Markov chain. The key intuition of our proposed methodology relies on the time-reversible nature of physical motion exhibited by most objects, which in turn should be possessed by a good tracker. In the presence of tracking failures due to occlusion, low SNR, or modeling errors, this reversible nature of the tracker is violated. We use this property for detection of track failures. To evaluate the performance of the tracker at time instant t, we use the posterior of the tracking algorithm to initialize a time-reversed Markov chain. We compute the posterior density of track parameters at the starting time t = 0 by filtering back in time to the initial time instant. The distance between the posterior density of the time-reversed chain (at t = 0) and the prior density used to initialize the tracking algorithm forms the decision statistic for evaluation. It is observed that when the data are generated by the underlying models, the decision statistic takes a low value. We provide a thorough experimental analysis of the evaluation methodology. Specifically, we demonstrate the effectiveness of our approach for tackling common challenges such as occlusion, pose, and illumination changes and provide the Receiver Operating Characteristic (ROC) curves. Finally, we also show the applicability of the core ideas of the paper to other tracking algorithms such as the Kanade-Lucas-Tomasi (KLT) feature tracker and the mean-shift tracker. Currently sparse signal reconstruction gains considerable interest and is applied in many fields. In this paper, a similarity induced by joint sparse representation is designed to construct the likelihood function of particle filter tracker so that the color visual spectrum and thermal spectrum images can be fused for object tracking. The proposed fusion scheme performs joint sparse representation calculation on both modalities and the resultant tracking results are fused using min operation on the sparse representation coefficients. In addition, a co-learning approach is proposed to update the reference templates of both modality and enhance the tracking robustness. The proposed fusion scheme outperforms state-of-the-art approaches, and its effectiveness is verified using OTCBVS database.