In this paper we propose a novel approach to multi-action recognition that performs joint segmentation and classification. This approach models each action using a Gaussian mixture using robust low-dimensional action features. Segmentation is achieved by performing classification on overlapping temporal windows, which are then merged to produce the final result. This approach is considerably less complicated than previous methods which use dynamic programming or computationally expensive hidden Markov models (HMMs). Initial experiments on a stitched version of the KTH dataset show that the proposed approach achieves an accuracy of 78.3 , outperforming a recent HMM-based approach which obtained 71.2 .
An approach termed Hidden Markov Model for Multiple, Irregular Observations (HMM-MIO) @cite_8 has also been proposed for the multi-action recognition task. HMM-MIO jointly segments and classifies observations which are irregular in time and space, and characterised by high dimensionality. The high dimensionality is reduced by using PPCA (probabilistic Principal Component Analysis). Moreover, HMM-MIO deals with heavy tails and outliers exhibited by empirical distributions by modelling the observation densities with a long-tailed distribution, the student's @math . HMM-MIO requires the search of the following 4 optimal parameters: (i) the resulting reduced dimension ( @math ), (ii) the number of components in each observation mixture ( @math ), (iii) the degree of the @math -distribution ( @math ), and (iv) the number of cells (or regions) used to deal with the space irregularity @math . To this end, for multiple observations of a frame, they postulate:
@cite_8
2019586054
Hidden Markov models (HMMs) provide joint segmentation and classification of sequential data by efficient inference algorithms and have therefore been employed in fields as diverse as speech recognition, document processing, and genomics. However, conventional HMMs do not suit action segmentation in video due to the nature of the measurements which are often irregular in space and time, high dimensional and affected by outliers. For this reason, in this paper we present a joint action segmentation and classification approach based on an extended model: the hidden Markov model for multiple, irregular observations (HMM-MIO). Experiments performed over a concatenated version of the popular KTH action dataset and the challenging CMU multi-modal activity dataset (CMU-MMAC) report accuracies comparable to or higher than those of a bag-of-features approach, showing the usefulness of improved sequential models for joint action segmentation and classification tasks.