Online signature verification technologies, such as those available in banks and post offices, rely on dedicated digital devices such as tablets or smart pens to capture, analyze and verify signatures. In this paper, we suggest a novel method for online signature verification that relies on the increasingly available hand-worn devices, such as smartwatches or fitness trackers, instead of dedicated ad-hoc devices. Our method uses a set of known genuine and forged signatures, recorded using the motion sensors of a hand-worn device, to train a machine learning classifier. Then, given the recording of an unknown signature and a claimed identity, the classifier can determine whether the signature is genuine or forged. In order to validate our method, it was applied on 1980 recordings of genuine and forged signatures that we collected from 66 subjects in our institution. Using our method, we were able to successfully distinguish between genuine and forged signatures with a high degree of accuracy (0.98 AUC and 0.05 EER).
Closer to the field of handwriting analysis, several recent studies have tried to use motion data collected from wearable devices to recognize different writing gestures such as inferring the letter written. For example, the authors of @cite_23 investigated the task of writing in the air with a mobile device. In @cite_3 , researchers suggested a platform for recognizing text written on a whiteboard using a smartwatch. In @cite_17 , researchers tried to infer letters written on a sheet of paper. With a totally different purpose in mind, the authors of @cite_12 , tried to detect the letters typed on a keyboard using a smartwatch. Similarly, in @cite_2 , researchers presented a new attack method that allows attackers to extract sensitive information such as credit card or phone access PIN codes from motion sensors in wearable devices.
@cite_3 @cite_23 @cite_2 @cite_12 @cite_17
2339997970 2080116634 2220702836 2082629283 1986707552
In this paper, we show how motion data collected with a smartwatch can be used to infer text written on a whiteboard. All data is gathered by a commercial, off the shelf smartwatch, without any additional hardware. We obtain single letters recognition rates of 94 in the first guess. Using three guesses, we recognize 99 of the letters. This result is achieved by transforming raw sensor measurements into a coordinate system reflecting the orientation of the whiteboard. Therefore, training data recorded on one whiteboard can be used on another even if the user is writing at different heights or body postures. We show that the built in microphone picks up the sounds caused by the pen which can help to segment the input into individual letters. Numerous sensors in modern mobile phones enable a range of people-centric applications. This paper envisions a system called PhonePoint Pen that uses the in-built accelerometer in mobile phones to recognize human writing. By holding the phone like a pen, a user should be able to write short messages or draw simple diagrams in the air. The acceleration due to hand gestures can be translated into geometric strokes, and recognized as characters. We prototype the PhonePoint Pen on the Nokia N95 platform, and evaluate it through real users. Results show that English characters can be identified with an average accuracy of 91.9 , if the users conform to a few reasonable constraints. Future work is focused on refining the prototype, with the goal of offering a new user-experience that complements keyboards and touch-screens. Wearable technologies are today on the rise, becoming more common and broadly available to mainstream users. In fact, wristband and armband devices such as smartwatches and fitness trackers already took an important place in the consumer electronics market and are becoming ubiquitous. By their very nature of being wearable, these devices, however, provide a new pervasive attack surface threatening users privacy, among others. In the meantime, advances in machine learning are providing unprecedented possibilities to process complex data efficiently. Allowing patterns to emerge from high dimensional unavoidably noisy data. The goal of this work is to raise awareness about the potential risks related to motion sensors built-in wearable devices and to demonstrate abuse opportunities leveraged by advanced neural network architectures. The LSTM-based implementation presented in this research can perform touchlogging and keylogging on 12-keys keypads with above-average accuracy even when confronted with raw unprocessed data. Thus demonstrating that deep neural networks are capable of making keystroke inference attacks based on motion sensors easier to achieve by removing the need for non-trivial pre-processing pipelines and carefully engineered feature extraction strategies. Our results suggest that the complete technological ecosystem of a user can be compromised when a wearable wristband device is worn. Imagine a user typing on a laptop keyboard while wearing a smart watch. This paper asks whether motion sensors from the watch can leak information about what the user is typing. While its not surprising that some information will be leaked, the question is how much? We find that when motion signal processing is combined with patterns in English language, the leakage is substantial. Reported results show that when a user types a word @math , it is possible to shortlist a median of 24 words, such that @math is in this shortlist. When the word is longer than @math characters, the median shortlist drops to @math . Of course, such leaks happen without requiring any training from the user, and also under the (obvious) condition that the watch is only on the left hand. We believe this is surprising and merits awareness, especially in light of various continuous sensing apps that are emerging in the app market. Moreover, we discover additional "leaks" that can further reduce the shortlist -- we leave these exploitations to future work. Smartwatch is becoming one of the most popular wearable device with many major smartphone manufacturers such as Samsung and Apple releasing their smartwatches recently. Apart from the fitness applications, the smartwatch provides a rich user interface that has enabled many applications like instant messaging and email. Since the smartwatch is worn on the wrist, it introduces a unique opportunity to understand user's arm, hand and possibly finger movements using its accelerometer and gyroscope sensors. Although user's arm and hand gestures are likely to be identified with ease using the smartwatch sensors, it is not clear how much of user's finger gestures can be recognized. In this paper, we show that motion energy measured at the smartwatch is sufficient to uniquely identify user's hand and finger gestures. We identify essential features of accelerometer and gyroscope data that reflect the movements of tendons (passing through the wrist) when performing a finger or a hand gesture. With these features, we build a classifier that can uniquely identify 37 (13 finger, 14 hand and 10 arm) gestures with an accuracy of 98 . We further extend our gesture recognition to identify the characters written by the user with her index finger on a surface, and show that such finger-writing can also be accurately recognized with nearly 95 accuracy. Our presented results will enable many novel applications like remote control and finger-writing-based input to devices using smartwatch.