This paper presents a novel approach in a rarely studied area of computer vision: Human interaction recognition in still images. We explore whether the facial regions and their spatial configurations contribute to the recognition of interactions. In this respect, our method involves extraction of several visual features from the facial regions, as well as incorporation of scene characteristics and deep features to the recognition. Extracted multiple features are utilized within a discriminative learning framework for recognizing interactions between people. Our designed facial descriptors are based on the observation that relative positions, size and locations of the faces are likely to be important for characterizing human interactions. Since there is no available dataset in this relatively new domain, a comprehensive new dataset which includes several images of human interactions is collected. Our experimental results show that faces and scene characteristics contain important information to recognize interactions between people.
One of the earliest works to recognize the human interactions in still images is the work of @cite_17 . In their paper, four classes are defined: shaking hands, pointing at the opposite person, standing hand-in-hand and intermediate-transitional state between them, and K-nearest neighbor classifier is used to recognize the interactions. Recently, @cite_16 has focused on how people interact by investigating the proxemics between them. They claim that complex interactions can be modeled as a single representation and a joint model of body poses can be learned. @cite_13 look into the problem of detecting social roles in videos in a weakly supervised setting via a CRF model. In our work, we approach the human-human interaction recognition problem by means of several descriptors that encode facial region configurations.
@cite_16 @cite_13 @cite_17
2164295580 1979545636 1974477890 2046589395
This paper presents a recognition system that classifies four kinds of human interactions: shaking hands, pointing at the opposite person, standing hand-in-hand, and an intermediate transitional state between them. Our system achieves recognition by applying the K-nearest neighbor classifier to the parametric human-interaction model, which describes the interpersonal configuration with multiple features from gray scale images (i.e., binary blob, silhouette contour, and intensity distribution). Unlike the algorithms that use temporal information about motion, our system independently classifies each frame by estimating the relative poses of the interacting persons. The system provides a tool to detect the initiation and the termination of an interaction with no parsing procedure for sequential data. Experimental results are presented and illustrated. The objective of this work is recognition and spatiotemporal localization of two-person interactions in video. Our approach is person-centric. As a first stage we track all upper bodies and heads in a video using a tracking-by-detection approach that combines detections with KLT tracking and clique partitioning, together with occlusion detection, to yield robust person tracks. We develop local descriptors of activity based on the head orientation (estimated using a set of pose-specific classifiers) and the local spatiotemporal region around them, together with global descriptors that encode the relative positions of people as a function of interaction type. Learning and inference on the model uses a structured output SVM which combines the local and global descriptors in a principled manner. Inference using the model yields information about which pairs of people are interacting, their interaction class, and their head orientation (which is also treated as a variable, enabling mistakes in the classifier to be corrected using global context). We show that inference can be carried out with polynomial complexity in the number of people, and describe an efficient algorithm for this. The method is evaluated on a new dataset comprising 300 video clips acquired from 23 different TV shows and on the benchmark UT--Interaction dataset. In this paper we propose a novel method to detect and record various posture-based events of interest in a typical elderly monitoring application in a home surveillance scenario. These events include standing, sitting, bending squatting, side lying and lying toward the camera. The projection histograms of segmented human body silhouette are used as the main feature vector for posture classification. k-nearest neighbor (k-NN) algorithm and evidence accumulation technique is proposed to infer human postures. With this technique we have achieved a robust recognition rate of above 90 and a stable classifier's output. The modified classifier structure also improves greatly the recognition rate of lying toward the camera events as compared to the result of classifier's structure in GHOST (, 1998). Furthermore, we use the speed of fall to differentiate real fall incident and an event where the person is simply lying without falling. Detecting objects in cluttered scenes and estimating articulated human body parts are two challenging problems in computer vision. The difficulty is particularly pronounced in activities involving human-object interactions (e.g. playing tennis), where the relevant object tends to be small or only partially visible, and the human body parts are often self-occluded. We observe, however, that objects and human poses can serve as mutual context to each other – recognizing one facilitates the recognition of the other. In this paper we propose a new random field model to encode the mutual context of objects and human poses in human-object interaction activities. We then cast the model learning task as a structure learning problem, of which the structural connectivity between the object, the overall human pose, and different body parts are estimated through a structure search approach, and the parameters of the model are estimated by a new max-margin algorithm. On a sports data set of six classes of human-object interactions [12], we show that our mutual context model significantly outperforms state-of-the-art in detecting very difficult objects and human poses.