CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> Background subtraction is a useful and effective method for detecting moving objects in video images. Since this method assumes that image variations are caused only by moving objects (i.e., the background scene is assumed to be stationary), however, its applicability is limited. In this paper, we propose a background subtraction method that robustly handles various changes in the background. The method learns the chronological changes in the observed scene's background in terms of distributions of image vectors. The method operates the subtraction by evaluating the Mahalanobis distances between the averages of such image vectors and newly observed image vectors. The method we propose herein expresses actual changes in the background using a multi-dimensional image vector space. This enables the method to detect objects with the correct sensitivity. We also introduce an eigenspace to reduce the computational cost. We describe herein how approximate Mahalanobis distances are obtained in this eigenspace. In our experiments, we confirmed the proposed method's effectiveness for real world scenes. <s> BIB001 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> We introduce a behavior-based similarity measure which tells us whether two different space-time intensity patterns of two different video segments could have resulted from a similar underlying motion field. This is done directly from the intensity information, without explicitly computing the underlying motions. Such a measure allows us to detect similarity between video segments of differently dressed people performing the same type of activity. It requires no foreground/background segmentation, no prior learning of activities, and no motion estimation or tracking. Using this behavior-based similarity measure, we extend the notion of 2-dimensional image correlation into the 3-dimensional space-time volume, thus allowing to correlate dynamic behaviors and actions. Small space-time video segments (small video clips) are "correlated" against entire video sequences in all three dimensions (x,y, and t). Peak correlation values correspond to video locations with similar dynamic behaviors. Our approach can detect very complex behaviors in video sequences (e.g., ballet movements, pool dives, running water), even when multiple complex activities occur simultaneously within the field-of-view of the camera. <s> BIB002 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> This paper explores the use of volumetric features for action recognition. First, we propose a novel method to correlate spatio-temporal shapes to video clips that have been automatically segmented. Our method works on over-segmented videos, which means that we do not require background subtraction for reliable object segmentation. Next, we discuss and demonstrate the complementary nature of shape- and flow-based features for action recognition. Our method, when combined with a recent flow-based correlation technique, can detect a wide range of actions in video, as demonstrated by results on a long tennis video. Although not specifically designed for whole-video classification, we also show that our method's performance is competitive with current action classification techniques on a standard video classification dataset. <s> BIB003 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> In this paper we propose a novel appearance descriptor for 3D human pose estimation from monocular images using a learning-based technique. Our image-descriptor is based on the intermediate local appearance descriptors that we design to encapsulate local appearance context and to be resilient to noise. We encode the image by the histogram of such local appearance context descriptors computed in an image to obtain the final image-descriptor for pose estimation. We name the final image-descriptor the Histogram of Local Appearance Context (HLAC). We then use Relevance Vector Machine (RVM) regression to learn the direct mapping between the proposed HLAC image-descriptor space and the 3D pose space. Given a test image, we first compute the HLAC descriptor and then input it to the trained regressor to obtain the final output pose in real time. We compared our approach with other methods using a synchronized video and 3D motion dataset. We compared our proposed HLAC image-descriptor with the Histogram of Shape Context and Histogram of SIFT like descriptors. The evaluation results show that HLAC descriptor outperforms both of them in the context of 3D Human pose estimation. <s> BIB004 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> Tracking human body poses in monocular video has many important applications. The problem is challenging in realistic scenes due to background clutter, variation in human appearance and self-occlusion. The complexity of pose tracking is further increased when there are multiple people whose bodies may inter-occlude. We proposed a three-stage approach with multi-level state representation that enables a hierarchical estimation of 3D body poses. Our method addresses various issues including automatic initialization, data association, self and inter-occlusion. At the first stage, humans are tracked as foreground blobs and their positions and sizes are coarsely estimated. In the second stage, parts such as face, shoulders and limbs are detected using various cues and the results are combined by a grid-based belief propagation algorithm to infer 2D joint positions. The derived belief maps are used as proposal functions in the third stage to infer the 3D pose using data-driven Markov chain Monte Carlo. Experimental results on several realistic indoor video sequences show that the method is able to track multiple persons during complex movement including sitting and turning movements with self and inter-occlusion. <s> BIB005 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> Action is any meaningful movement of the human and it is used to convey information or to interact naturally without any mechanical devices. Human action recognition is motivated by some of the applications such as video retrieval, Human robot interaction, to interact with deaf and dumb people etc. In any Action Recognition System, some pre-processing steps are done for removing the noise caused because of illumination effects, blurring, false contour etc. Background subtraction is done to remove the static or slowly varying background. In this paper, multiple background subtraction algorithms are tested and then one of them is selected for the further process of action recognition. Background subtraction is also known as foreground/background segmentation or foreground extraction. The next step is the feature extraction which deals with the extraction of the important feature (like corner points, optical flow, shape, motion vectors etc.) from the image frame. The proposed novel action recognition algorithm uses discrete Fourier transform (DFT) of the small image block. <s> BIB006 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> Human action recognition is gaining interest from many computer vision researchers because of its wide variety of potential applications. For instance: surveillance, advanced human computer interaction, content-based video retrieval, or athletic performance analysis. In this research, we focus to recognize some human actions such as waving, punching, clapping, etc. We choose exemplar-based sequential single-layered approach using Dynamic Time Warping (DTW) because of its robustness against variation in speed or style in performing action. For improving recognition rate, we perform body part tracking using depth camera to recover human joints body part information in 3D real world coordinate system. We build our feature vector from joint orientation along time series that invariant to human body size. Dynamic Time Warping is then applied to the resulted feature vector. We examine our approach to recognize several actions and we confirm our method can work well with several experiments. Further experiment for benchmarking the result will be held in near future. <s> BIB007 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> This paper proposed an approach for human body part tracking which is based on torso tracking. The main goal in this paper is tracking main human body parts such as torso, head and hands. In the proposed method for human body part tracking, we are using connected components to improve the detected silhouette (i.e. detected foreground using background subtraction) in order to detect the body parts with respect to torso location and size. We are also using a blob tracking module which is composed of foreground detection, blob detection and blob tracking in order to find the approximate location and size of torso in each frame (i.e. torso tracking). By tracking torso, we will be able to track other body parts based on their location with respect to the torso. Having found torso size and location, the region of certain body parts on the silhouette will be modeled by an Gaussian ellipse using Gaussian blob modeling in each frame with different color in order to show its location, size and pose. The proposed HBPT approach helps us robustly identify and track human body parts in real-time for further use in Human Activity Recognition in vision-based HCI. <s> BIB008 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> This paper proposes a system to recognize quasi-periodic human actions from monocular video sequences. First, each input video frame is analyzed and estimated to generate the best 3D human model pose which consists of a set of 3D coordinates of specific human joints. Next, these 3D coordinates for each frame are converted into corresponding 3D geometric relational features (GRFs), which describe the geometric relations among body joints of a pose. Finally, we train a cyclic hidden Markov model (CHMM) for each action based on the vector quantized 3D GRFs, and the trained CHMMs are used to classify different quasi-periodic human actions. The experimental results indicate the effectiveness of the proposed system in terms of the view point invariance, the low-dimensional feature vectors, and the encouraging recognition rates. <s> BIB009 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> Human actions can be represented by the trajectories of skeleton joints. Traditional methods generally model the spatial structure and temporal dynamics of human skeleton with hand-crafted features and recognize human actions by well-designed classifiers. In this paper, considering that recurrent neural network (RNN) can model the long-term contextual information of temporal sequences well, we propose an end-to-end hierarchical RNN for skeleton based action recognition. Instead of taking the whole skeleton as the input, we divide the human skeleton into five parts according to human physical structure, and then separately feed them to five subnets. As the number of layers increases, the representations extracted by the subnets are hierarchically fused to be the inputs of higher layers. The final representations of the skeleton sequences are fed into a single-layer perceptron, and the temporally accumulated output of the perceptron is the final decision. We compare with five other deep RNN architectures derived from our model to verify the effectiveness of the proposed network, and also compare with several other methods on three publicly available datasets. Experimental results demonstrate that our model achieves the state-of-the-art performance with high computational efficiency. <s> BIB010 </s> CPS-Based Smart Warehouse for Industry 4.0: A Survey of the Underlying Technologies <s> Vision-Based Human Activity Recognition <s> This work targets human action recognition in video. While recent methods typically represent actions by statistics of local video features, here we argue for the importance of a representation derived from human pose. To this end we propose a new Pose-based Convolutional Neural Network descriptor (P-CNN) for action recognition. The descriptor aggregates motion and appearance information along tracks of human body parts. We investigate different schemes of temporal aggregation and experiment with P-CNN features obtained both for automatically estimated and manually annotated human poses. We evaluate our method on the recent and challenging JHMDB and MPII Cooking datasets. For both datasets our method shows consistent improvement over the state of the art. <s> BIB011
For vision-based human activity recognition, there are mainly three steps: (1) Human Segmentation; (2) Feature Extraction; and (3) Activity Recognition. These three procedures are the basis for vision-based human activity recognition. There are higher level scenarios and applications that require specific setting and modeling, for instance, single-person or multi-person scenario, gesture recognition, human identification, etc. The first step, human segmentation, is to segment the human object from the images or a sequence of video screen-shoots. Common methods for human segmentation is to apply background segmentation  BIB001 on the images and foreground human objects are extracted. Then, feature extraction is performed to describe the characteristics and movements of the segmented human objects. Different categories of features can be leveraged as features, including space-time volume BIB003 BIB002 , frequency BIB006 , local descriptors BIB004 and body modeling BIB008 BIB005 . The last step relies on the activity recognition and classification algorithms to recognize different kinds of human activities based on the features extracted above. Traditional, Dynamic Time Warping BIB007 , Hidden Markov Model BIB009 and some machine learning algorithms, e.g., SVM (Support Vector Machine) and ANN (Artificial Neural Network), are applied for activity recognition. In recent years, with the development of deep learning technologies, activity recognition has been realized with greater results. For example, Recurrent Neural Network BIB010 and Convolutional Neural Network BIB011 have been applied in pose recognition.