We present SMAUG (Secure Mobile Authentication Using Gestures), a novel biometric assisted authentication algorithm for mobile devices that is solely based on data collected from multiple sensors that are usually installed on modern devices -- touch screen, gyroscope and accelerometer. As opposed to existing approaches, our system supports a fully flexible user input such as free-form gestures, multi-touch, and arbitrary amount of strokes. Our experiments confirm that this approach provides a high level of robustness and security. More precisely, in 77 of all our test cases over all gestures considered, a user has been correctly identified during the first authentication attempt and in 99 after the third attempt, while an attacker has been detected in 97 of all test cases. As an example, gestures that have a good balance between complexity and usability, e.g., drawing a two parallel lines using two fingers at the same time, 100 success rate after three login attempts and 97 impostor detection rate were given. We stress that we consider the strongest possible attacker model: an attacker is not only allowed to monitor the legitimate user during the authentication process, but also receives additional information on the biometric properties, for example pressure, speed, rotation, and acceleration. We see this method as a significant step beyond existing authentication methods that can be deployed directly to devices in use without the need of additional hardware.
With respect to gesture recognition for single-touch gestures, Rubine @cite_30 is the usual reference when comparing new single-touch algorithms. Another prominent example for single-touch and single-stroke gesture recognition is @cite_0 . The authors of @cite_36 present a very efficient follow-up work for single-touch and multi-stroke. In short, the algorithm joins all strokes in all possible combinations and reduces the gesture recognition problem therefore to the case of single-touch gestures. However, this algorithm family needs a predefined set of gestures. The authors in @cite_1 have developed a multi-dimension DTW for gesture recognition. In @cite_35 , the authors present a gesture based user authentication scheme for touch screens using solely the accelerometer. 3D hand gesture recognition in the air with mobile devices and accelerometer is examined in @cite_15 . Similar research was done for Kinect and gesture recognition in @cite_27 , also for Wii @cite_38 .
@cite_30 @cite_35 @cite_38 @cite_36 @cite_1 @cite_0 @cite_27 @cite_15
2595328592 2097384738 1963581134 2134917076
Gesture recognition aims to recognize meaningful movements of human bodies, and is of utmost importance in intelligent human–computer robot interactions. In this paper, we present a multimodal gesture recognition method based on 3-D convolution and convolutional long-short-term-memory (LSTM) networks. The proposed method first learns short-term spatiotemporal features of gestures through the 3-D convolutional neural network, and then learns long-term spatiotemporal features by convolutional LSTM networks based on the extracted short-term spatiotemporal features. In addition, fine-tuning among multimodal data is evaluated, and we find that it can be considered as an optional skill to prevent overfitting when no pre-trained models exist. The proposed method is verified on the ChaLearn LAP large-scale isolated gesture data set (IsoGD) and the Sheffield Kinect gesture (SKIG) data set. The results show that our proposed method can obtain the state-of-the-art recognition accuracy (51.02 on the validation set of IsoGD and 98.89 on SKIG). We present a new framework for multimodal gesture recognition that is based on a multiple hypotheses rescoring fusion scheme. We specifically deal with a demanding Kinect-based multimodal data set, introduced in a recent gesture recognition challenge (ChaLearn 2013), where multiple subjects freely perform multimodal gestures. We employ multiple modalities, that is, visual cues, such as skeleton data, color and depth images, as well as audio, and we extract feature descriptors of the hands' movement, handshape, and audio spectral properties. Using a common hidden Markov model framework we build single-stream gesture models based on which we can generate multiple single stream-based hypotheses for an unknown gesture sequence. By multimodally rescoring these hypotheses via constrained decoding and a weighted combination scheme, we end up with a multimodally-selected best hypothesis. This is further refined by means of parallel fusion of the monomodal gesture models applied at a segmental level. In this setup, accurate gesture modeling is proven to be critical and is facilitated by an activity detection system that is also presented. The overall approach achieves 93.3 gesture recognition accuracy in the ChaLearn Kinect-based multimodal data set, significantly outperforming all recently published approaches on the same challenging multimodal gesture recognition task, providing a relative error rate reduction of at least 47.6 . In most applications of touch based humancomputer interaction, multi-touch gestures are used for directlymanipulating the interface such as scaling, panning, etc. In thispaper, we propose using multi-touch gesture as indirectcommand, such as redo, undo, erase, etc., for the operatingsystem. The proposed recognition system is guided by temporal,spatial and shape information. This is achieved using a graphembedding approach where all previous information are used.We evaluated our multi-touch recognition system on a set of 18different multi-touch gestures. With this graph embeddingmethod and a SVM classifier, we achieve 94.50 recognition rate.We believe that our research points out a possibility ofintegrating together raw ink, direct manipulation and indirectcommand in many gesture-based complex application such as asketch drawing application. In many applications today user interaction is moving away from mouse and pens and is becoming pervasive and much more physical and tangible. New emerging interaction technologies allow developing and experimenting with new interaction methods on the long way to providing intuitive human computer interaction. In this paper, we aim at recognizing gestures to interact with an application and present the design and evaluation of our sensor-based gesture recognition. As input device we employ the Wii-controller (Wiimote) which recently gained much attention world wide. We use the Wiimote's acceleration sensor independent of the gaming console for gesture recognition. The system allows the training of arbitrary gestures by users which can then be recalled for interacting with systems like photo browsing on a home TV. The developed library exploits Wii-sensor data and employs a hidden Markov model for training and recognizing user-chosen gestures. Our evaluation shows that we can already recognize gestures with a small number of training samples. In addition to the gesture recognition we also present our experiences with the Wii-controller and the implementation of the gesture recognition. The system forms the basis for our ongoing work on multimodal intuitive media browsing and are available to other researchers in the field.