Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> This paper describes the challenging efforts of glo bal security in vision system along with the judgment of the display expression in the design map of accessible human computer interaction systems. Our framework is desi gned for vision-based videos, which contains many instances of a target face from a sequence of slightly different views. The vertex mask generation is based on facia l feature point detection according to its geometrical, angling and position constraint s. This approach relies on our real time face modeling technique, by which the individu al face model is created using an original vectorize model at different views of a fa ce. In the characteristics model, we label the face features by their maximum and minimu m curvatures. Finally, analyze the performance efficiency of the database containing 47 people’s faces with at least 5 different expression under un iform lighting conditions demonstrate significant performance achievements. Vision-based Security system is designed and develo ped for the security based system that allow efficient and natural phenomenon for both communication and interaction at any conceivable environment. Vision based system circled around W5H phenomenon which is being exposed by considering the identification, ac tivity detection (input), recognition, time limitation according to appropriate place, accuracy (output) and emotion control are discussed in our research. Simultaneously, human behavior monitoring is not only verified by everyday experience but also the validity of comprehensive i nformation play side-by-side role in realtime communication environment. While, comprehensive set of effective access control between vision ubiquitous architecture with secure database leads towards strong and everbuild communication. Further, ubiquitous computing along with wireless network fetches secure information of human computer interaction by the allocation of different video cameras position. In this paper we present identifi cation applications which represented the feature vectors of fixed frontal, by applying the c ritical boundary algorithm to identify unknown face. While in verification application, th e numerical implementations meet the need of validity of the claimed identity to conside r the output matching score with likelihood ratio phenomenon. Successful implementation of face verification move towards the face recognition section which evaluates different techn iques to get the most matching scores eignfaces. Finally, management query information of particular input (eigenface) is authenticate, schedule, monitor and serve as an out put. Therefore, our goal is to convey viewer’s attention towards different matters to pre serves the global security and reliability of our system. According to comparison of our architecture with ot her researchers, pattern matching methods deal with huge number of dimensions feature vector with their redundant characteristics. So the recognition is not accurate and hardly to make stable. Due to this condition, some lowdimensional feature vector is required to give sure ty of different eigenfaces. Also view <s> BIB001 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> This paper presents the standardized effects of GMHS (Global Mobile Home Security) technology to achieve high security reasons using transmission of data (queries and data calls) in cellular networks. GMHS make use of the mobile's SMS and 3G services conducting the two-way communication between the primary user and the Security device. This operation is conducted by verifying the primary cellular number along with the security pin code and the query. The device has been made capable to alert user, of any change occurring intentionally and unintentionally, making it more secure and friendly. The results show the performance with respect to service conditions as the service is mainly depend upon the cellular network being used as the medium for queries. <s> BIB002 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> This paper describes the security feature of e-learning authentication. The main goal of this research is to achieve authentication to identify legal user. Internet is open for all users to access and share information. Simultaneously, the user hackers are also their, to examine a web application and infrastructure to understand its design, identify the potentially weak aspects, and use these weaknesses to break or exploit the application. Through authentication process, we could overcome the illegal usage of application. We proposed different algorithms for authentication that is RIPEMD -160. RIPEMD-160 capture overall secure authentication. <s> BIB003 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> We propose a distributed recognition method to classify human actions using a low-bandwidth wearable motion sensor network. Given a set of pre-segmented motion sequences as training examples, the algorithm simultaneously segments and classifies human actions, and it also rejects outlying actions that are not in the training set. The classification is distributedly operated on individual sensor nodes and a base station computer. We show that the distribution of multiple action classes satisfies a mixture subspace model, one sub-space for each action class. Given a new test sample, we seek the sparsest linear representation of the sample w.r.t. all training examples. We show that the dominant coefficients in the representation only correspond to the action class of the test sample, and hence its membership is encoded in the representation. We further provide fast linear solvers to compute such representation via l1-minimization. Using up to eight body sensors, the algorithm achieves state-of-the-art 98.8% accuracy on a set of 12 action categories. We further demonstrate that the recognition precision only decreases gracefully using smaller subsets of sensors, which validates the robustness of the distributed framework. <s> BIB004 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> We present a framework for the automatic recognition of complex multi-agent events in settings where structure is imposed by rules that agents must follow while performing activities. Given semantic spatio-temporal descriptions of what generally happens (i.e., rules, event descriptions, physical constraints), and based on video analysis, we determine the events that occurred. Knowledge about spatio-temporal structure is encoded using first-order logic using an approach based on Allen's Interval Logic, and robustness to low-level observation uncertainty is provided by Markov Logic Networks (MLN). Our main contribution is that we integrate interval-based temporal reasoning with probabilistic logical inference, relying on an efficient bottom-up grounding scheme to avoid combinatorial explosion. Applied to one-on-one basketball, our framework detects and tracks players, their hands and feet, and the ball, generates event observations from the resulting trajectories, and performs probabilistic logical inference to determine the most consistent sequence of events. We demonstrate our approach on 1hr (100,000 frames) of outdoor videos. <s> BIB005 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Hand gesture recognition systems for virtual reality applications provides the users an enhanced interaction experience as it integrates the virtual and the real world object. Growth in virtual environments based upon computer systems and development of user interfaces influence the changes in the Human-Computer Interaction (HCI). Gesture recognition based interaction interface, endow with more realistic and immersive interaction compared to the traditional devices. The system enables a physically realistic mode of interaction to the virtual environment. The Hand gesture recognition system based interface proposed and implemented in this paper consists of a detection, tracking and recognition module. For the implementation of these modules various image processing algorithms as Camshift, Lucas Kanade, Haar like features etc has been employed. Comprehensive user acceptability has been considered to exhibit the accuracy, usefulness and ease of use to the proposed and implemented hand gesture recognition system. Hand gesture communication based vocabulary offers many variations ranging from simple action of using our finger to point at to using hands for moving objects around to the rather complex one like expression of the feelings. The proposed hand gesture recognition system offers intensions to traditional input devices for interaction with the virtual environments. The gesture based interaction interface being proposed here can be substantially applied towards many applications like Virtual Reality, Sign Language and Games. Though the present paper considered games as the application domain. <s> BIB006 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> We present a human activity recognition (HAR) system for smart homes utilizing depth silhouettes and R transformation. Previously, R transformation has been applied only on binary silhouettes which provide only the shape information of human activities. In this work, we utilize R transformation on depth silhouettes such that the depth information of human body parts can be used in HAR in addition to the shape information. In R transformation, 2D directional projection maps are computed through Radon transform, and then 1D feature profiles, that are translation and scaling invariant, are computed through R transform. Then, we apply Principle Component Analysis and Linear Discriminant Analysis to extract prominent activity features. Finally, Hidden Markov Models are used to train and recognize daily home activities. Our results show the mean recognition rate of 96.55% over ten typical home activities whereas the same system utilizing binary silhouettes achieves only 85.75%. Our system should be useful as a smart HAR system for smart homes. <s> BIB007 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Entertainment and gaming systems such as the Wii and XBox Kinect have brought touchless, body-movement based interfaces to the masses. Systems like these enable the estimation of movements of various body parts from raw inertial motion or depth sensor data. However, the interface developer is still left with the challenging task of creating a system that recognizes these movements as embodying meaning. The machine learning approach for tackling this problem requires the collection of data sets that contain the relevant body movements and their associated semantic labels. These data sets directly impact the accuracy and performance of the gesture recognition system and should ideally contain all natural variations of the movements associated with a gesture. This paper addresses the problem of collecting such gesture datasets. In particular, we investigate the question of what is the most appropriate semiotic modality of instructions for conveying to human subjects the movements the system developer needs them to perform. The results of our qualitative and quantitative analysis indicate that the choice of modality has a significant impact on the performance of the learnt gesture recognition system; particularly in terms of correctness and coverage. <s> BIB008 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> The goal of high-level event recognition is to automatically detect complex high-level events in a given video sequence. This is a difficult task especially when videos are captured under unconstrained conditions by non-professionals. Such videos depicting complex events have limited quality control, and therefore, may include severe camera motion, poor lighting, heavy background clutter, and occlusion. However, due to the fast growing popularity of such videos, especially on the Web, solutions to this problem are in high demands and have attracted great interest from researchers. In this paper, we review current technologies for complex event recognition in unconstrained videos. While the existing solutions vary, we identify common key modules and provide detailed descriptions along with some insights for each of them, including extraction and representation of low-level features across different modalities, classification strategies, fusion techniques, etc. Publicly available benchmark datasets, performance metrics, and related research forums are also described. Finally, we discuss promising directions for future research. <s> BIB009 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> This paper provides a comprehensive survey for activity recognition in video surveillance. It starts with a description of simple and complex human activity, and various applications. The applications of activity recognition are manifold, ranging from visual surveillance through content based retrieval to human computer interaction. The organization of this paper covers all aspects of the general framework of human activity recognition. Then it summarizes and categorizes recent-published research progresses under a general framework. Finally, this paper also provides an overview of benchmark databases for activity recognition, the market analysis of video surveillance, and future directions to work on for this application. <s> BIB010 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> In this paper, we describe how to detect abnormal human activities taking place in an outdoor surveillance environment. Human tracks are provided in real time by the baseline video surveillance system. Given trajectory information, the event analysis module will attempt to determine whether or not a suspicious activity is currently being observed. However, due to real-time processing constrains, there might be false alarms generated by video image noise or non-human objects. It requires further intensive examination to filter out false event detections which can be processed in an off-line fashion. We propose a hierarchical abnormal event detection system that takes care of real time and semi-real time as multi-tasking. In low level task, a trajectory-based method processes trajectory data and detects abnormal events in real time. In high level task, an intensive video analysis algorithm checks whether the detected abnormal event is triggered by actual humans or not. <s> BIB011 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> The TREC Video Retrieval Evaluation (TRECVID) 2012 was a TREC-style video analysis and retrieval evaluation, the goal of which remains to promote progress in content-based exploitation of digital video via open, metrics-based evaluation. Over the last ten years this effort has yielded a better understanding of how systems can effectively accomplish such processing and how one can reliably benchmark their performance. TRECVID is funded by the NIST and other US government agencies. Many organizations and individuals worldwide contribute significant time and effort. <s> BIB012 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Providing accurate and opportune information on people's activities and behaviors is one of the most important tasks in pervasive computing. Innumerable applications can be visualized, for instance, in medical, security, entertainment, and tactical scenarios. Despite human activity recognition (HAR) being an active field for more than a decade, there are still key aspects that, if addressed, would constitute a significant turn in the way people interact with mobile devices. This paper surveys the state of the art in HAR based on wearable sensors. A general architecture is first presented along with a description of the main components of any HAR system. We also propose a two-level taxonomy in accordance to the learning approach (either supervised or semi-supervised) and the response time (either offline or online). Then, the principal issues and challenges are discussed, as well as the main solutions to each one of them. Twenty eight systems are qualitatively evaluated in terms of recognition performance, energy consumption, obtrusiveness, and flexibility, among others. Finally, we present some open problems and ideas that, due to their high relevance, should be addressed in future research. <s> BIB013 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Human activity recognition (HAR) is an emerging methodology essential for smart homes with practical applications such as personal lifecare and healthcare services for the elderly and disabled people. In this work, we present a novel HAR methodology utilizing the recognized body parts of human depth silhouettes and Hidden Markov Models (HMMs). We first create a database of synthetic depth silhouettes and their corresponding body parts labelled silhouettes of various human activities to train random forests (RFs). With the trained RFs, a set of 23 body parts are recognized from incoming depth silhouettes, yielding a set of centroids from the identified body parts. From the dynamics of these centroids, motion parameters are computed: a set of magnitude and directional angle features. Finally, the spatio-temporal dynamics of these motion features of various activities are used to train HMMs. We have performed HAR with the trained HMMs for six typical home activities and obtained the mean recognition rate of 97.16%. The presented HAR methodology should be useful for residents monitoring services at smart homes. <s> BIB014 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> AIMS AND OBJECTIVES: To investigate the dynamics of domestic abuse awareness and recognition among primary healthcare professionals and abused women. BACKGROUND: Domestic abuse is a serious, public health issue that crosses geographical and demographic boundaries. Health professionals are well placed to recognise and respond to domestic abuse, but empirical evidence suggests that they are reluctant to broach the issue. Moreover, research has shown that women are reluctant to disclose abuse. DESIGN: A two-phase, qualitative study was conducted in Scotland. METHODS: Twenty-nine primary health professionals (midwives, health visitors and general practitioners) participated in the first phase of the study, and 14 abused women took part in phase two. Data were collected in 2011. Semi-structured, individual interviews were conducted with the health professionals, and three focus groups were facilitated with the abused women. Data were analysed using a framework analysis approach. FINDINGS: Differing levels of awareness of the nature and existence of abuse are held by abused women and primary healthcare professionals. Specifically, many women do not identify their experiences as abusive. A conceptual representation of domestic abuse - the "abused women, awareness, recognition and empowerment' framework - arising from the study - presents a new way of capturing the complexity of the disclosure process. CONCLUSION: Further research is necessary to test and empirically validate the framework, but it has potential pedagogical use for the training and education of health professionals and clinical use with abused women. RELEVANCE TO CLINICAL PRACTICE: The framework may be used in clinical practice by nurses and other health professionals to facilitate open discussion between professionals and women. In turn, this may empower women to make choices regarding disclosure and safety planning. Language: en <s> BIB015 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> The Internet technology is creating huge opportunities to expand existing businesses and forming what is called New Economy, Global Economy, or Electronic-Commerce (E-Commerce). E-Commerce describes business transactions, customer services, ordering, delivery and payment, and intra-business tasks that make use of the public internet and the digital networked computing environment that links organizations and individuals in business, industry, government, and the home. However, many organizations are intimidated by the new technologies, unsure of how to take advantage of them, and wondering how these technologies will support existing investments in skills and infrastructures. In addition, this new type of economy or commerce comes with a lot of challenges especially those related to trust and security issues. In this paper, the different types of security issues facing e-commerce systems will be presented and categorized, in addition, general guidelines and measures on how to deal with these security issues to protect e-commerce systems will be presented and discussed. <s> BIB016 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Human action recognition in videos is useful for many applications. However, there still exist huge challenges in real applications due to the variations in the appearance, lighting condition and viewing angle, of the subjects. In this consideration, depth data have advantages over red, green, blue (RGB) data because of their spatial information about the distance between object and viewpoint. Unlike existing works, we utilize the 3-D point cloud, which contains points in the 3-D real-world coordinate system to represent the external surface of human body. Specifically, we propose a new robust feature, the body surface context (BSC), by describing the distribution of relative locations of the neighbors for a reference point in the point cloud in a compact and descriptive way. The BSC encodes the cylindrical angular of the difference vector based on the characteristics of human body, which increases the descriptiveness and discriminability of the feature. As the BSC is an approximate object-centered feature, it is robust to transformations including translations and rotations, which are very common in real applications. Furthermore, we propose three schemes to represent human actions based on the new feature, including the skeleton-based scheme, the random-reference-point scheme, and the spatial-temporal scheme. In addition, to evaluate the proposed feature, we construct a human action dataset by a depth camera. Experiments on three datasets demonstrate that the proposed feature outperforms RGB-based features and other existing depth-based features, which validates that the BSC feature is promising in the field of human action recognition. <s> BIB017 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> This paper presents two sets of features, shape representation and kinematic structure, for human activity recognition using a sequence of RGB-D images. The shape features are extracted using the depth information in the frequency domain via spherical harmonics representation. The other features include the motion of the 3D joint positions (i.e. the end points of the distal limb segments) in the human body. Both sets of features are fused using the Multiple Kernel Learning (MKL) technique at the kernel level for human activity recognition. Our experiments on three publicly available datasets demonstrate that the proposed features are robust for human activity recognition and particularly when there are similarities among the actions. <s> BIB018 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Recent advancements in depth video sensors technologies have made human activity recognition (HAR) realizable for elderly monitoring applications. Although conventional HAR utilizes RGB video sensors, HAR could be greatly improved with depth video sensors which produce depth or distance information. In this paper, a depth-based life logging HAR system is designed to recognize the daily activities of elderly people and turn these environments into an intelligent living space. Initially, a depth imaging sensor is used to capture depth silhouettes. Based on these silhouettes, human skeletons with joint information are produced which are further used for activity recognition and generating their life logs. The life-logging system is divided into two processes. Firstly, the training system includes data collection using a depth camera, feature extraction and training for each activity via Hidden Markov Models. Secondly, after training, the recognition engine starts to recognize the learned activities and produces life logs. The system was evaluated using life logging features against principal component and independent component features and achieved satisfactory recognition rates against the conventional approaches. Experiments conducted on the smart indoor activity datasets and the MSRDailyActivity3D dataset show promising results. The proposed system is directly applicable to any elderly monitoring system, such as monitoring healthcare problems for elderly people, or examining the indoor activities of people at home, office or hospital. <s> BIB019 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Face Recognition begins with extracting the coordinates of features such as width of mouth, width of eyes, pupil, and compare the result with the measurements stored in the database and return the closest record (facial metrics). Nowadays, there are a lot of face recognition techniques and algorithms found and developed around the world. Facial recognition becomes an interesting research topic. It is proven by numerous number of published papers related with facial recognition including facial feature extraction, facial algorithm improvements, and facial recognition implementations. Main purposes of this research are to get the best facial recognition algorithm (Eigenface and Fisherface) provided by the Open CV 2.4.8 by comparing the ROC (Receiver Operating Characteristics) curve and implement it in the attendance system as the main case study. Based on the experiments, the ROC curve proves that using the current training set, Eigenface achieves better result than Fisherface. Eigenface implemented inside the Attendance System returns between 70% to 90% similarity for genuine face images. <s> BIB020 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Abstract Recently still image-based human action recognition has become an active research topic in computer vision and pattern recognition. It focuses on identifying a person׳s action or behavior from a single image. Unlike the traditional action recognition approaches where videos or image sequences are used, a still image contains no temporal information for action characterization. Thus the prevailing spatiotemporal features for video-based action analysis are not appropriate for still image-based action recognition. It is more challenging to perform still image-based action recognition than the video-based one, given the limited source of information as well as the cluttered background for images collected from the Internet. On the other hand, a large number of still images exist over the Internet. Therefore it is demanding to develop robust and efficient methods for still image-based action recognition to understand the web images better for image retrieval or search. Based on the emerging research in recent years, it is time to review the existing approaches to still image-based action recognition and inspire more efforts to advance the field of research. We present a detailed overview of the state-of-the-art methods for still image-based action recognition, and categorize and describe various high-level cues and low-level features for action analysis in still images. All related databases are introduced with details. Finally, we give our views and thoughts for future research. <s> BIB021 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Abstract Human activity recognition has been an important area of computer vision research since the 1980s. Various approaches have been proposed with a great portion of them addressing this issue via conventional cameras. The past decade has witnessed a rapid development of 3D data acquisition techniques. This paper summarizes the major techniques in human activity recognition from 3D data with a focus on techniques that use depth data. Broad categories of algorithms are identified based upon the use of different features. The pros and cons of the algorithms in each category are analyzed and the possible direction of future research is indicated. <s> BIB022 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Detecting anomalies in data is a vital task, with numerous high-impact applications in areas such as security, finance, health care, and law enforcement. While numerous techniques have been developed in past years for spotting outliers and anomalies in unstructured collections of multi-dimensional points, with graph data becoming ubiquitous, techniques for structured graph data have been of focus recently. As objects in graphs have long-range correlations, a suite of novel technology has been developed for anomaly detection in graph data. This survey aims to provide a general, comprehensive, and structured overview of the state-of-the-art methods for anomaly detection in data represented as graphs. As a key contribution, we give a general framework for the algorithms categorized under various settings: unsupervised versus (semi-)supervised approaches, for static versus dynamic graphs, for attributed versus plain graphs. We highlight the effectiveness, scalability, generality, and robustness aspects of the methods. What is more, we stress the importance of anomaly attribution and highlight the major techniques that facilitate digging out the root cause, or the `why', of the detected anomalies for further analysis and sense-making. Finally, we present several real-world applications of graph-based anomaly detection in diverse domains, including financial, auction, computer traffic, and social networks. We conclude our survey with a discussion on open theoretical and practical challenges in the field. <s> BIB023 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> This paper is a postprint of a paper submitted to and accepted for publication in IET Computer Vision and is subject to Institution of Engineering and Technology Copyright. The copy of record is available at IET Digital Library and at IEEE Xplore. <s> BIB024 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> In this growing age of technology it is necessary to have a proper health care management system which should be cent percent accurate but also should be portable so that every person carry with it as personalized health care system. The health care management system which will consist of mobile based Heart Rate Measurement so that the data can be transferred and diagnosis based on heart rate can be provided quickly with a click of button. The system will consist of video conferencing to connect remotely with the Doctor. The Doc-Bot which was developed earlier is now being transferred to mobile platform and will be further advanced for diagnosis of common diseases. The system will also consist of Online Blood Bank which will provide up-to-date details about availability of blood in different hospitals. <s> BIB025 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Recent development in depth sensors opens up new challenging task in the field of computer vision research areas, including human-computer interaction, computer games and surveillance systems. This paper addresses shape and motion features approach to observe, track and recognize human silhouettes using a sequence of RGB-D images. Under our proposed activity recognition framework, the required procedure includes: detecting human silhouettes from the image sequence, we remove noisy effects from background and track human silhouettes using temporal continuity constraints of human motion information for each activity, extracting the shape and motion features to identify richer motion information and then these features are clustered and fed into Hidden Markov Model (HMM) to train, model and recognize human activities based on transition and emission probabilities values. During experimental results, we demonstrate this approach on two challenging depth video datasets: one based on our own annotated database and other based on public database (i.e., MSRAction3D). Our approach shows significant recognition results over the state of the art algorithms. <s> BIB026 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> This paper addresses the issues of 3D human activity detection, tracking and recognition from RGB-D video sequences using a feature structured framework. During human tracking and activity recognition, initially, dense depth images are captured using depth camera. In order to track human silhouettes, we considered spatial/temporal continuity, constraints of human motion information and compute centroids of each activity based on chain coding mechanism and centroids point extraction. In body skin joints features, we estimate human body skin color to identify human body parts (i.e., head, hands, and feet) likely to extract joint points information. These joints points are further processed as feature extraction process including distance position features and centroid distance features. Lastly, self-organized maps are used to recognize different activities. Experimental results demonstrate that the proposed method is reliable and efficient in recognizing human poses at different realistic scenes. The proposed system should be applicable to different consumer application systems such as healthcare system, video surveillance system and indoor monitoring systems which track and recognize different activities of multiple users. <s> BIB027 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Text recognition in images is a research area which attempts to develop a computer system with the ability to automatically read the text from images. These days there is a huge demand in storing the information available in paper documents format in to a computer storage disk and then later reusing this information by searching process. One simple way to store information from these paper documents in to computer system is to first scan the documents and then store them as images. But to reuse this information it is very difficult to read the individual contents and searching the contents form these documents line-by-line and word-by-word. The challenges involved in this the font characteristics of the characters in paper documents and quality of images. Due to these challenges, computer is unable to recognize the characters while reading them. Thus there is a need of character recognition mechanisms to perform Document Image Analysis (DIA) which transforms documents in paper format to electronic format. In this paper we have discuss method for text recognition from images. The objective of this paper is to recognition of text from image for better understanding of the reader by using particular sequence of different processing module. <s> BIB028 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Activity and action detection, tracking and recognition are very demanding research area in computer vision and human computer interaction. In this paper, a video-based novel approach for human activity recognition is presented using robust hybrid features and embedded Hidden Markov Models. In the proposed HAR framework, depth maps are analyzed by temporal motion identification method to segment human silhouettes from noisy background and compute depth silhouette area for each activity to track human movements in a scene. Several representative features, including invariant, depth sequential silhouettes and spatiotemporal body joints features were fused together to explore gradient orientation change, intensity differentiation, temporal variation and local motion of specific body parts. Then, these features are processed by the dynamics of their respective class and learned, trained and recognized with specific embedded HMM having active feature values. Our experiments on two depth datasets demonstrate that the proposed features are efficient and robust over the state of the arts features for human activity recognition especially when there are similar postures of different activities. <s> BIB029 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Abstract This paper presents an overview of state-of-the-art methods in activity recognition using semantic features. Unlike low-level features, semantic features describe inherent characteristics of activities. Therefore, semantics make the recognition task more reliable especially when the same actions look visually different due to the variety of action executions. We define a semantic space including the most popular semantic features of an action namely the human body (pose and poselet), attributes, related objects, and scene context. We present methods exploiting these semantic features to recognize activities from still images and video data as well as four groups of activities: atomic actions, people interactions, human–object interactions, and group activities. Furthermore, we provide potential applications of semantic approaches along with directions for future research. <s> BIB030 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Action recognition has become a very important topic in computer vision with unconstrained video sequences. There are varieties of approaches to feature extraction and video sequences description, which play important roles in action recognition. In this paper, we survey the main representations along dense trajectories and aggregating methods for the videos in the last decade. We mainly discuss the aggregating methods which are bag of words (BOW), fisher vector (FV) and vector of locally aggregated descriptors (VLAD). Furthermore, the newest mean average precision (mAP) obtained from the references is used to discuss different aggregating methods on realistic datasets. And for more intuitive comparison those aggregating methods, we will evaluate them on KTH in the same conditions. Finally, we analyze and compare those papers' experimental data to summarize the trends. Based on the reviews from several approaches to action recognition, we further make an analysis and discussion on the technical trends in this field. <s> BIB031 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> In this paper, we propose to use the deep learning technique for abnormal event detection by extracting spatiotemporal features from video sequences. Human eyes are often attracted to abnormal events in video sequences, thus we firstly extract saliency information (SI) of video frames as the feature representation in the spatial domain. Optical flow (OF) is estimated as an important feature of video sequences in the temporal domain. To extract the accurate motion information, multi-scale histogram optical flow (MHOF) can be obtained through OF. We combine MHOF and SI into the spatiotemporal features of video frames. Finally a deep learning network, PCANet, is adopted to extract high-level features for abnormal event detection. Experimental results show that the proposed abnormal event detection method can obtain much better performance than the existing ones on the public video database. <s> BIB032 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> In recent years, the task of event recognition from videos has attracted increasing interest in multimedia area. While most of the existing research was mainly focused on exploring visual cues to handle relatively small-granular events, it is difficult to directly analyse video content without any prior knowledge. Therefore, synthesizing both the visual and semantic analysis is a natural way for video event understanding. In this paper, we study the problem of web video event recognition, where web videos often describe largegranular events and carry limited textual information. Key challenges include how to accurately represent event semantics from incomplete textual information and how to effectively explore the correlation between visual and textual cues for video event understanding. We propose a novel framework to perform complex event recognition from web videos. In order to compensate the insufficient expressive power of visual cues, we construct an event knowledge base by deeply mining semantic information from ubiquitous web documents. This event knowledge base is capable of describing each event with comprehensive semantics. By utilizing this base, the textual cues for a video can be significantly enriched. Furthermore, we introduce a two-view adaptive regression model which explores the intrinsic correlation between the visual and textual cues of the videos to learn reliable classifiers. Extensive experiments on two real-world video datasets show the effectiveness of our proposed framework and prove that the event knowledge base indeed helps improve the performance of web video event recognition. <s> BIB033 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> ABSTRACTA family of Nyquist-I pulses called sinc parametric linear combination pulse (SPLCP) is proposed. It is characterized by two novel design parameters that provide additional degrees of freedom to minimize the intercarrier interference (ICI) power due to frequency offset. Moreover, it reduces the high peak-to-average power ratio (PAPR) value in orthogonal frequency division multiplexing (OFDM) systems. Several Nyquist-I pulses were recently proposed to address the subject of high sensitivity to frequency offset and high PAPR in OFDM-based transmissions. In this paper, we investigate the performance of SPLCP in terms of ICI power, signal-to-interference ratio (SIR) power, bit error rate (BER), and PAPR. We additionally examine the behaviour of SPLCP with new design parameters for a certain roll-off factor, α. We compare the performance of SPLCP with other well-known pulses. Theoretical and simulation results show that the proposed SPLCP outperforms other existing pulses in terms of ICI power, SIR pow... <s> BIB034 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Human activity recognition using depth information is an emerging and challenging technology in computer vision due to its considerable attention by many practical applications such as smart home/office system, personal health care and 3D video games. This paper presents a novel framework of 3D human body detection, tracking and recognition from depth video sequences using spatiotemporal features and modified HMM. To detect human silhouette, raw depth data is examined to extract human silhouette by considering spatial continuity and constraints of human motion information. While, frame differentiation is used to track human movements. Features extraction mechanism consists of spatial depth shape features and temporal joints features are used to improve classification performance. Both of these features are fused together to recognize different activities using the modified hidden Markov model (M-HMM). The proposed approach is evaluated on two challenging depth video datasets. Moreover, our system has significant abilities to handle subject"s body parts rotation and body parts missing which provide major contributions in human activity recognition. <s> BIB035 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Human action recognition is an important branch among the studies of both human perception and computer vision systems. Along with the development of artificial intelligence, deep learning techniques have gained remarkable reputation when dealing with image categorization tasks (e.g., object detection and classification). However, since human actions normally present in the form of sequential image frames, analyzing human action data requires significantly increased computational power than still images when deep learning techniques are employed. Such a challenge has been the bottleneck for the migration of learning-based image representation techniques to action sequences, so that the old fashioned handcrafted human action representations are still widely used for human action recognition tasks. On the other hand, since handcrafted representations are usually ad-hoc and overfit to specific data, they are incapable of being generalized to deal with various realistic scenarios. Consequently, resorting to deep learning action representations for human action recognition tasks is eventually a natural option. In this work, we provide a detailed overview of recent advancements in human action representations. As the first survey that covers both handcrafted and learning-based action representations, we explicitly discuss the superiorities and limitations of exiting techniques from both kinds. The ultimate goal of this survey is to provide comprehensive analysis and comparisons between learning-based and handcrafted action representations respectively, so as to inspire action recognition researchers towards the study of both kinds of representation techniques. We provide a comprehensive stud over the state-of-the-art action representations.Deep learning-based representations are compared to handcrafted representations.Pros and cons of current deep learning-based approaches are discussed. <s> BIB036 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Understanding human actions in visual data is tied to advances in complementary research areas including object recognition, human dynamics, domain adaptation and semantic segmentation. Over the last decade, human action analysis evolved from earlier schemes that are often limited to controlled environments to nowadays advanced solutions that can learn from millions of videos and apply to almost all daily activities. Given the broad range of applications from video surveillance to human-computer interaction, scientific milestones in action recognition are achieved more rapidly, eventually leading to the demise of what used to be good in a short time. This motivated us to provide a comprehensive review of the notable steps taken towards recognizing human actions. To this end, we start our discussion with the pioneering methods that use handcrafted representations, and then, navigate into the realm of deep learning based approaches. We aim to remain objective throughout this survey, touching upon encouraging improvements as well as inevitable fallbacks, in the hope of raising fresh questions and motivating new research directions for the reader. <s> BIB037 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> In this paper, we present a unified approach for abnormal behavior detection and group behavior analysis in video scenes. Existing approaches for abnormal behavior detection do either use trajectory-based or pixel-based methods. Unlike these approaches, we propose an integrated pipeline that incorporates the output of object trajectory analysis and pixel-based analysis for abnormal behavior inference. This enables to detect abnormal behaviors related to speed and direction of object trajectories, as well as complex behaviors related to finer motion of each object. By applying our approach on three different data sets, we show that our approach is able to detect several types of abnormal group behaviors with less number of false alarms compared with existing approaches. <s> BIB038 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> The Kinect sensing devices have been widely used in current Human-Computer Interaction entertainment. A fundamental issue involved is to detect users' motions accurately and quickly. In this paper, we tackle it by proposing a linear algorithm, which is augmented by feature interaction. The linear property guarantees its speed whereas feature interaction captures the higher order effect from the data to enhance its accuracy. The Schatten-p norm is leveraged to integrate the main linear effect and the higher order nonlinear effect by mining the correlation between them. The resulted classification model is a desirable combination of speed and accuracy. We propose a novel solution to solve our objective function. Experiments are performed on three public Kinect-based entertainment data sets related to fitness and gaming. The results show that our method has its advantage for motion detection in a real-time Kinect entertaining environment. <s> BIB039 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Nurses regularly perform patient handling activities. These activities with awkward postures expose healthcare providers to a high risk of overexertion injury. The recognition of patient handling activities is the first step to reduce injury risk for caregivers. The current practice on workplace activity recognition is based on human observational approach, which is neither accurate nor projectable to a large population. In this paper, we aim at addressing these challenges. Our solution comprises a smart wearable device and a novel spatio-temporal warping (STW) pattern recognition framework. The wearable device, named Smart Insole 2.0, is equipped with a rich set of sensors and can provide an unobtrusive way to automatically capture the information of patient handling activities. The STW pattern recognition framework fully exploits the spatial and temporal characteristics of plantar pressure by calculating a novel warped spatio-temporal distance, to quantify the similarity for the purpose of activity recognition. To validate the effectiveness of our framework, we perform a pilot study with eight subjects, including eight common activities in a nursing room. The experimental results show the overall classification accuracy achieves 91.7%. Meanwhile, the qualitative profile and load level can also be classified with accuracies of 98.3% and 92.5%, respectively. <s> BIB040 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> In this paper, we propose new approaches for action and event recognition by leveraging a large number of freely available Web videos (e.g., from Flickr video search engine) and Web images (e.g., from Bing and Google image search engines). We address this problem by formulating it as a new multi-domain adaptation problem, in which heterogeneous Web sources are provided. Specifically, we are given different types of visual features (e.g., the DeCAF features from Bing/Google images and the trajectory-based features from Flickr videos) from heterogeneous source domains and all types of visual features from the target domain. Considering the target domain is more relevant to some source domains, we propose a new approach named multi-domain adaptation with heterogeneous sources (MDA-HS) to effectively make use of the heterogeneous sources. In MDA-HS, we simultaneously seek for the optimal weights of multiple source domains, infer the labels of target domain samples, and learn an optimal target classifier. Moreover, as textual descriptions are often available for both Web videos and images, we propose a novel approach called MDA-HS using privileged information (MDA-HS+) to effectively incorporate the valuable textual information into our MDA-HS method, based on the recent learning using privileged information paradigm. MDA-HS+ can be further extended by using a new elastic-net-like regularization. We solve our MDA-HS and MDA-HS+ methods by using the cutting-plane algorithm, in which a multiple kernel learning problem is derived and solved. Extensive experiments on three benchmark data sets demonstrate that our proposed approaches are effective for action and event recognition without requiring any labeled samples from the target domain. <s> BIB041 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> In this paper we propose a novel learning framework called Supervised and Weakly Supervised Learning where the goal is to learn simultaneously from weakly and strongly labeled data. Strongly labeled data can be simply understood as fully supervised data where all labeled instances are available. In weakly supervised learning only data is weakly labeled which prevents one from directly applying supervised learning methods. Our proposed framework is motivated by the fact that a small amount of strongly labeled data can give considerable improvement over only weakly supervised learning. The primary problem domain focus of this paper is acoustic event and scene detection in audio recordings. We first propose a naive formulation for leveraging labeled data in both forms. We then propose a more general framework for Supervised and Weakly Supervised Learning (SWSL). Based on this general framework, we propose a graph based approach for SWSL. Our main method is based on manifold regularization on graphs in which we show that the unified learning can be formulated as a constraint optimization problem which can be solved by iterative concave-convex procedure (CCCP). Our experiments show that our proposed framework can address several concerns of audio content analysis using weakly labeled data. <s> BIB042 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Multimedia event detection has been one of the major endeavors in video event analysis. A variety of approaches have been proposed recently to tackle this problem. Among others, using semantic representation has been accredited for its promising performance and desirable ability for human-understandable reasoning. To generate semantic representation, we usually utilize several external image/video archives and apply the concept detectors trained on them to the event videos. Due to the intrinsic difference of these archives, the resulted representation is presumable to have different predicting capabilities for a certain event. Notwithstanding, not much work is available for assessing the efficacy of semantic representation from the source-level. On the other hand, it is plausible to perceive that some concepts are noisy for detecting a specific event. Motivated by these two shortcomings, we propose a bi-level semantic representation analyzing method. Regarding source-level, our method learns weights of semantic representation attained from different multimedia archives. Meanwhile, it restrains the negative influence of noisy or irrelevant concepts in the overall concept-level. In addition, we particularly focus on efficient multimedia event detection with few positive examples, which is highly appreciated in the real-world scenario. We perform extensive experiments on the challenging TRECVID MED 2013 and 2014 datasets with encouraging results that validate the efficacy of our proposed approach. <s> BIB043 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Abstract The recently developed depth imaging technologies have provided new directions for human activity recognition (HAR) without attaching optical markers or any other motion sensors to human body parts. In this paper, we propose novel multi-fused features for online human activity recognition (HAR) system that recognizes human activities from continuous sequences of depth map. The proposed online HAR system segments human depth silhouettes using temporal human motion information as well as it obtains human skeleton joints using spatiotemporal human body information. Then, it extracts the spatiotemporal multi-fused features that concatenate four skeleton joint features and one body shape feature. Skeleton joint features include the torso-based distance feature (DT), the key joint-based distance feature (DK), the spatiotemporal magnitude feature (M) and the spatiotemporal directional angle feature ( θ ). The body shape feature called HOG-DDS represents the projections of the depth differential silhouettes (DDS) between two consecutive frames onto three orthogonal planes by the histogram of oriented gradients (HOG) format. The size of the proposed spatiotemporal multi-fused feature is reduced by a code vector in the code book which is generated by vector quantization method. Then, it trains the hidden Markov model (HMM) with the code vectors of the multi-fused features and recognizes the segmented human activity by the forward spotting scheme using the trained HMM-based human activity classifiers. The experimental results on three challenging depth video datasets such as IM-DailyDepthActivity, MSRAction3D and MSRDailyActivity3D demonstrate that the proposed online HAR method using the proposed multi-fused features outperforms the state-of-the-art HAR methods in terms of recognition accuracy. <s> BIB044 </s> Feature Encodings and Poolings for Action and Event Recognition: A Comprehensive Survey <s> Introduction <s> Video-based human action recognition has become one of the most popular research areas in the field of computer vision and pattern recognition in recent years. It has a wide variety of applications such as surveillance, robotics, health care, video searching and human-computer interaction. There are many challenges involved in human action recognition in videos, such as cluttered backgrounds, occlusions, viewpoint variation, execution rate, and camera motion. A large number of techniques have been proposed to address the challenges over the decades. Three different types of datasets namely, single viewpoint, multiple viewpoint and RGB-depth videos, are used for research. This paper presents a review of various state-of-the-art deep learning-based techniques proposed for human action recognition on the three types of datasets. In light of the growing popularity and the recent developments in video-based human action recognition, this review imparts details of current trends and potential directions for future work to assist researchers. <s> BIB045
More and more research efforts within the computer vision community have focused on recognizing actions and events from uncontrolled videos over the past two decades. There are many promising applications for action and event recognition, such as abnormal action and event recognition in surveillance applications BIB038 BIB024 BIB011 BIB032 , interaction action and event recognition in entertainment applications BIB039 BIB005 BIB006 BIB008 , and home-based rehabilitation action and event recognition in healthcare applications BIB040 BIB025 BIB007 BIB015 , and many other analogous applications such as in BIB041 BIB033 BIB003 BIB016 BIB001 BIB002 . According to the definition given by NIST BIB012 , an event is a complex activity occurring at a specific place and time, which involves people interacting with other people and/or objects, and consists of a number of human actions, processes, and activities. Feature representation approaches, pattern recognition models and performance evaluation strategies are three key components in action and event recognition BIB009 . Compared to the other two key components, feature representation approaches, which should contain robust appearance and motion information, played a more critical role in video analysis. There are three key components in feature representation approaches, which are feature extractions, feature encodings, and feature poolings. Feature extractions care mainly about how to extract required features from specified multimedia collections. On the one hand, there are many approaches proposed for feature extraction using specified sensors in previous research work. For example, Lara et al. BIB013 surveyed the state of the art in wearable sensors-based human activity recognition, and categorized feature extraction from time series data into statistical approaches and structural approaches, with measured attributes of acceleration, environmental signals and vital signs. Jalal et al.  developed a life logging system, where both magnitude features and direction angle features were extracted from depth silhouettes of human activities captured by a depth camera. Yang et al. BIB004 applied a low-bandwidth wearable motion sensor network to recognize human actions distributed on individual sensor nodes and a base station computer, based on a set of LDA features. Song et al. BIB017 proposed a robust feature approach, namely the body surface context, to encode the cylindrical angular of the difference vector according to the characteristics of human body, for action recognition from videos of depth camera. Jalal et al. BIB014 presented a methodology for human activity recognition-based smart home application, by extraction of both magnitude features and direction features from human silhouettes in a depth camera. Althloothi et al. BIB018 extracted two sets of 3D spatiotemporal features via a Kinect sensor for human activity recognition, where the shape features were from the surface points using spherical harmonics coefficients, and the motion features were from the end points of the distal limb segments. Besides, similar research work in the area can be found in references BIB026 BIB019 BIB034 . On the other hand, there are also a number of approaches proposed for extraction of multimedia features in previous research work, including audio features, visual features, text features, and hybrid features. For example, Li et al.  proposed the extraction of deep audio features for acoustic event detection by a multi-stream hierarchical deep neural network. Kumar et al. BIB042 proposed a unified approach that adopted strongly and weakly labeled data for audio event and scene recognition based on a mel-ceptra coefficients feature. Farooq et al. BIB027 constructed a feature-structured framework by using skin joints features and self-organizing map for 3D human activity detection, tracking and recognition from RGB-D video sequences. Siswanto et al. BIB020 verified by experiments that PCA-based Eigenface outperformed LDA-based Fisherface for facial recognition for biometrics-based time attendance purposes. Manwatkar et al. BIB028 designed an automatic image-text recognition system by using matrix features and Kohonen neural networks. Chang et al. BIB043 invented a source-concept bi-level semantic representation analysis framework for multimedia event detection. Jalal et al. BIB029 proposed a hybrid feature representation approach called depth silhouettes context, which is fused of invariant features, depth sequential silhouettes features and spatiotemporal body joints features, for human activity recognition based on embedded Hidden Markov Models. Kamal et al. BIB035 presented a framework for 3D human body detection, tracking and recognition from depth video sequences using spatiotemporal features and modified HMM. Jalal et al. BIB044 proposed a hybrid multi-fused spatiotemporal feature representation approach that concatenated four skeleton joint features and one body shape feature to recognize human activity from depth video. The feature encodings and poolings concern primarily how to organize effectively the extracted features, so that overall performances in action and event recognition, such as recognition precision, recognition robustness and computation complexity, can be further improved. The difference between action representation and event representation lies mainly in their feature extraction approaches, whereas these two representations usually share same feature encodings and poolings. Due to complex spatiotemporal relationships between actions and events in multimedia collections, feature encodings and poolings are becoming increasingly important in feature representation approaches. There have been many survey papers available on feature representation approaches of action and event recognition BIB010 BIB021 BIB022 BIB030 BIB023  BIB031 BIB036 BIB045 BIB037 . However, most of them focused on reviewing proposed feature extraction approaches, and thus did not take into comprehensive account feature encoding and pooling approaches, which have been widely used in both image and video analysis. The purpose of the paper is to conduct a complete survey on the most popular feature encoding and pooling approaches in action and event recognition from recent years.