The recent trend in action recognition is towards larger datasets, an increasing number of action classes and larger visual vocabularies. State-of-the-art human action classification in challenging video data is currently based on a bag-of-visual-words pipeline in which space-time features are aggregated globally to form a histogram. The strategies chosen to sample features and construct a visual vocabulary are critical to performance, in fact often dominating performance. In this work we provide a critical evaluation of various approaches to building a vocabulary and show that good practises do have a significant impact. By subsampling and partitioning features strategically, we are able to achieve state-of-the-art results on 5 major action recognition datasets using relatively small visual vocabularies.
bof per category The selection of good partitioning clusters to form a visual vocabulary is also important, as they form the basic units of the histogram representation on which a classifier will base its decision @cite_12 . Thus, for a categorisation task, having clusters which represent feature patches that distinguish the classes is most likely to make classification easier. This motivates per-category clustering, to preserve discriminative information that may be lost by a universal vocabulary @cite_12 , especially when distinct categories are very similar. The downside is that learning a separate visual vocabulary per class may also generate many redundant clusters when features are shared amongst multiple categories. On the other hand, since the complexity of building visual vocabularies depends on the number of cluster centres @math , clustering features independently allows to reduce @math whilst keeping the representation dimensionality high, and makes the vocabulary learning easily parallelisable. So far, per-category training has seen promise on a single dataset with a small number of classes @cite_12 ; it is therefore to be seen how it performs on challenging action classification data with a large number of action classes.
@cite_12
280632315
In this paper we propose two distinct enhancements to the basic “bag-of-keypoints” image categorisation scheme proposed in [4]. In this approach images are represented as a variable sized set of local image features (keypoints). Thus, we require machine learning tools which can operate on sets of vectors. In [4] this is achieved by representing the set as a histogram over bins found by k-means. We show how this approach can be improved and generalised using Gaussian Mixture Models (GMMs). Alternatively, the set of keypoints can be represented directly as a probability density function, over which a kernel can be defined. This approach is shown to give state of the art categorisation performance.