Detecting carried objects is one of the requirements for developing systems to reason about activities involving people and objects. We present an approach to detect carried objects from a single video frame with a novel method that incorporates features from multiple scales. Initially, a foreground mask in a video frame is segmented into multi-scale superpixels. Then the human-like regions in the segmented area are identified by matching a set of extracted features from superpixels against learned features in a codebook. A carried object probability map is generated using the complement of the matching probabilities of superpixels to human-like regions and background information. A group of superpixels with high carried object probability and strong edge support is then merged to obtain the shape of the carried object. We applied our method to two challenging datasets, and results show that our method is competitive with or better than the state-of-the-art.
Aforementioned methods rely on a precise foreground segmentation which is often difficult to achieve in video surveillance due to sudden change of lighting or poor contrast from the surrounding environment. To overcome this problem, @cite_35 described a pedestrian by motion statistics using Gaussian mixture motion model and calculating short-term and long-term uniformity of motion. Short-term uniformity of motion models the periodic motion of limbs, and long-term uniformity of motion models uniform motion and corresponds to head and torso. If a person carries an object, her uniform motion profile does not fit to the average motion profile of unencumbered people. Those parts of a person that do not fit with both models are considered as carried objects.
@cite_35
2133217984
The availability of dense motion information in computer vision domain allows for the effective application of Lagrangian techniques that have their origin in fluid flow analysis and dynamical systems theory. A well established technique that has been proven to be useful in image-based crowd analysis are Finite Time Lyapunov Exponents (FTLE). Based on this, we present a method to detect people carrying object and describe a methodology how to apply established flow field methods onto the problem of describing individuals. Further, we reinterpret Lagrangian features in relation to the underlying motion process and show their applicability towards the appearance modeling of pedestrians. This definition allows to increase performance of state-of-the-art methods and is shown to be robust under varying parameter settings and different optical flow extraction approaches.