Aerial robots are becoming popular among general public, and with the development of artificial intelligence (AI), there is a trend to equip aerial robots with a natural user interface (NUI). Hand arm gestures are an intuitive way to communicate for humans, and various research works have focused on controlling an aerial robot with natural gestures. However, the techniques in this area are still far from mature. Many issues in this area have been poorly addressed, such as the principles of choosing gestures from the design point of view, hardware requirements from an economic point of view, considerations of data availability, and algorithm complexity from a practical perspective. Our work focuses on building an economical monocular system particularly designed for gesture-based piloting of an aerial robot. Natural arm gestures are mapped to rich target directions and convenient fine adjustment is achieved. Practical piloting scenarios, hardware cost and algorithm applicability are jointly considered in our system design. The entire system is successfully implemented in an aerial robot and various properties of the system are tested.
Static gestures are usually recognized by various hand-crafted features and rule-based algorithms @cite_27 @cite_12 @cite_26 @cite_19 @cite_7 @cite_21 @cite_16 @cite_13 @cite_23 @cite_5 , and dynamic gestures are typically recognized by state-space models @cite_0 @cite_19 @cite_21 . A neural-networks-based method is used in @cite_24 for image interpretation, predicting the angles of the arm from an image segment. We focus on static gesture recognition and the recognition results of multiple frames are summarized together to robustly generate a command. Our recognition algorithm is rule-based and is not data-driven.
@cite_26 @cite_7 @cite_21 @cite_0 @cite_19 @cite_27 @cite_24 @cite_23 @cite_5 @cite_16 @cite_13 @cite_12
2219948256 2914992758 2145835757 2119656522
This paper discusses the development of a natural gesture user interface that tracks and recognizes in real time hand gestures based on depth data collected by a Kinect sensor. The interest space corresponding to the hands is first segmented based on the assumption that the hand of the user is the closest object in the scene to the camera. A novel algorithm is proposed to improve the scanning time in order to identify the first pixel on the hand contour within this space. Starting from this pixel, a directional search algorithm allows for the identification of the entire hand contour. The @math -curvature algorithm is then employed to locate the fingertips over the contour, and dynamic time warping is used to select gesture candidates and also to recognize gestures by comparing an observed gesture with a series of prerecorded reference gestures. The comparison of results with state-of-the-art approaches shows that the proposed system outperforms most of the solutions for the static recognition of sign digits and is similar in terms of performance for the static and dynamic recognition of popular signs and for the sign language alphabet. The solution simultaneously deals with static and dynamic gestures as well as with multiple hands within the interest space. An average recognition rate of 92.4 is achieved over 55 static and dynamic gestures. Two possible applications of this work are discussed and evaluated: one for interpretation of sign digits and gestures for a friendlier human-machine interaction and the other one for the natural control of a software interface. Dynamic hand gesture recognition is a crucial yet challenging task in computer vision. The key of this task lies in an effective extraction of discriminative spatial and temporal features to model the evolutions of different gestures. In this paper, we propose an end-to-end Spatial-Temporal Attention Residual Temporal Convolutional Network (STA-Res-TCN) for skeleton-based dynamic hand gesture recognition, which learns different levels of attention and assigns them to each spatial-temporal feature extracted by the convolution filters at each time step. The proposed attention branch assists the networks to adaptively focus on the informative time frames and features while exclude the irrelevant ones that often bring in unnecessary noise. Moreover, our proposed STA-Res-TCN is a lightweight model that can be trained and tested in an extremely short time. Experiments on DHG-14 28 Dataset and SHREC’17 Track Dataset show that STA-Res-TCN outperforms state-of-the-art methods on both the 14 gestures setting and the more complicated 28 gestures setting. We introduce a discriminative hidden-state approach for the recognition of human gestures. Gesture sequences often have a complex underlying structure, and models that can incorporate hidden structures have proven to be advantageous for recognition tasks. Most existing approaches to gesture recognition with hidden states employ a Hidden Markov Model or suitable variant (e.g., a factored or coupled state model) to model gesture streams; a significant limitation of these models is the requirement of conditional independence of observations. In addition, hidden states in a generative model are selected to maximize the likelihood of generating all the examples of a given gesture class, which is not necessarily optimal for discriminating the gesture class against other gestures. Previous discriminative approaches to gesture sequence recognition have shown promising results, but have not incorporated hidden states nor addressed the problem of predicting the label of an entire sequence. In this paper, we derive a discriminative sequence model with a hidden state structure, and demonstrate its utility both in a detection and in a multi-way classification formulation. We evaluate our method on the task of recognizing human arm and head gestures, and compare the performance of our method to both generative hidden state and discriminative fully-observable models. This paper presents a novel and real-time system for interaction with an application or video game via hand gestures. Our system includes detecting and tracking bare hand in cluttered background using skin detection and hand posture contour comparison algorithm after face subtraction, recognizing hand gestures via bag-of-features and multiclass support vector machine (SVM) and building a grammar that generates gesture commands to control an application. In the training stage, after extracting the keypoints for every training image using the scale invariance feature transform (SIFT), a vector quantization technique will map keypoints from every training image into a unified dimensional histogram vector (bag-of-words) after K-means clustering. This histogram is treated as an input vector for a multiclass SVM to build the training classifier. In the testing stage, for every frame captured from a webcam, the hand is detected using our algorithm, then, the keypoints are extracted for every small image that contains the detected hand gesture only and fed into the cluster model to map them into a bag-of-words vector, which is finally fed into the multiclass SVM training classifier to recognize the hand gesture.