We present WiGest: a system that leverages changes in WiFi signal strength to sense in-air hand gestures around the user's mobile device. Compared to related work, WiGest is unique in using standard WiFi equipment, with no modi-fications, and no training for gesture recognition. The system identifies different signal change primitives, from which we construct mutually independent gesture families. These families can be mapped to distinguishable application actions. We address various challenges including cleaning the noisy signals, gesture type and attributes detection, reducing false positives due to interfering humans, and adapting to changing signal polarity. We implement a proof-of-concept prototype using off-the-shelf laptops and extensively evaluate the system in both an office environment and a typical apartment with standard WiFi access points. Our results show that WiGest detects the basic primitives with an accuracy of 87.5 using a single AP only, including through-the-wall non-line-of-sight scenarios. This accuracy in-creases to 96 using three overheard APs. In addition, when evaluating the system using a multi-media player application, we achieve a classification accuracy of 96 . This accuracy is robust to the presence of other interfering humans, highlighting WiGest's ability to enable future ubiquitous hands-free gesture-based interaction with mobile devices.
Gesture recognition systems generally adopt various techniques such as computer vision @cite_19 , inertial sensors @cite_35 , ultra-sonic @cite_5 , and infrared electromagnetic radiation (e.g. on Samsung S4). While promising, these techniques suffer from limitations such as sensitivity to lighting, high installation and instrumentation overhead, demanding dedicated sensors to be worn or installed, or requiring line-of-sight communication between the user and the sensor. These limitations promoted exploiting WiFi, already installed on most user devices and abundant within infrastructure, for and recognition as detailed in this section.
@cite_19 @cite_5 @cite_35
1982297923 2008110769 2595328592 2094645604
In this paper we present zSense, which provides greater input expressivity for spatially limited devices such as smart wearables through a shallow depth gesture recognition system using non-focused infrared sensors. To achieve this, we introduce a novel Non-linear Spatial Sampling (NSS) technique that significantly cuts down the number of required infrared sensors and emitters. These can be arranged in many different configurations; for example, number of sensor emitter units can be as minimal as one sensor and two emitters. We implemented different configurations of zSense on smart wearables such as smartwatches, smartglasses and smart rings. These configurations naturally fit into the flat or curved surfaces of such devices, providing a wide scope of zSense enabled application scenarios. Our evaluations reported over 94.8 gesture recognition accuracy across all configurations. In this paper we propose a hand gesture recognition system that relies on color and depth images, and on a small pose sensor on the human palm. Monocular and stereo vision systems have been used for human pose and gesture recognition, but with limited scope due to limitations on texture, illumination, etc. New RGB-Depth sensors, that reply on projected light such as the Microsoft Kinect, have overcome many of those limitations. However, the point clouds for hand gestures are still in many cases noisy and partially occluded, and hand gesture recognition is not trivial. Hand gesture recognition is much more complex than full body motion, since we can have the hands in any orientation and can not assume a standing body on a ground plane. In this work we propose to add a tiny pose sensor to the human palm, with a minute accelerometer and magnetometer that combined provide 3D angular pose, to reduce the search space and have a robust and computationally light recognition method. Starting with the full depth image point cloud, segmentation can be performed by taking into account the relative depth and hand orientation, as well as skin color. Identification is then performed by matching 3D voxel occupancy against a gesture template database. Preliminary results are presented for the recognition of Portuguese Sign Language alphabet, showing the validity of the approach. Gesture recognition aims to recognize meaningful movements of human bodies, and is of utmost importance in intelligent human–computer robot interactions. In this paper, we present a multimodal gesture recognition method based on 3-D convolution and convolutional long-short-term-memory (LSTM) networks. The proposed method first learns short-term spatiotemporal features of gestures through the 3-D convolutional neural network, and then learns long-term spatiotemporal features by convolutional LSTM networks based on the extracted short-term spatiotemporal features. In addition, fine-tuning among multimodal data is evaluated, and we find that it can be considered as an optional skill to prevent overfitting when no pre-trained models exist. The proposed method is verified on the ChaLearn LAP large-scale isolated gesture data set (IsoGD) and the Sheffield Kinect gesture (SKIG) data set. The results show that our proposed method can obtain the state-of-the-art recognition accuracy (51.02 on the validation set of IsoGD and 98.89 on SKIG). This paper presents a literature review on the use of depth for hand tracking and gesture recognition. The survey examines 37 papers describing depth-based gesture recognition systems in terms of (1) the hand localization and gesture classification methods developed and used, (2) the applications where gesture recognition has been tested, and (3) the effects of the low-cost Kinect and OpenNI software libraries on gesture recognition research. The survey is organized around a novel model of the hand gesture recognition process. In the reviewed literature, 13 methods were found for hand localization and 11 were found for gesture classification. 24 of the papers included real-world applications to test a gesture recognition system, but only 8 application categories were found (and three applications accounted for 18 of the papers). The papers that use the Kinect and the OpenNI libraries for hand tracking tend to focus more on applications than on localization and classification methods, and show that the OpenNI hand tracking method is good enough for the applications tested thus far. However, the limitations of the Kinect and other depth sensors for gesture recognition have yet to be tested in challenging applications and environments.