Bimanual gestures are of the utmost importance for the study of motor coordination in humans and in everyday activities. A reliable detection of bimanual gestures in unconstrained environments is fundamental for their clinical study and to assess common activities of daily living. This paper investigates techniques for a reliable, unconstrained detection and classification of bimanual gestures. It assumes the availability of inertial data originating from the two hands arms, builds upon a previously developed technique for gesture modelling based on Gaussian Mixture Modelling (GMM) and Gaussian Mixture Regression (GMR), and compares different modelling and classification techniques, which are based on a number of assumptions inspired by literature about how bimanual gestures are represented and modelled in the brain. Experiments show results related to 5 everyday bimanual activities, which have been selected on the basis of three main parameters: (not) constraining the two hands by a physical tool, (not) requiring a specific sequence of single-hand gestures, being recursive (or not). In the best performing combination of modeling approach and classification technique, five out of five activities are recognized up to an accuracy of 97 , a precision of 82 and a level of recall of 100 .
Wearable systems for the automatic recognition of human gestures and full-body movements typically rely on inertial information. Accelerometers prove to be the most informative sensor for the task @cite_9 . To comply with end users' constraints related to the impact of the monitoring system on their appearance and freedom of motion, most solutions adopt a single sensing device, either located at the waist @cite_7 or, as it is becoming more and more common, at the wrist @cite_2 .
@cite_9 @cite_7 @cite_2
1491797991 1968194699 2112589867
Smartwatch is becoming one of the most popular wearable device with many major smartphone manufacturers such as Samsung and Apple releasing their smartwatches recently. Apart from the fitness applications, the smartwatch provides a rich user interface that has enabled many applications like instant messaging and email. Since the smartwatch is worn on the wrist, it introduces a unique opportunity to understand user's arm, hand and possibly finger movements using its accelerometer and gyroscope sensors. Although user's arm and hand gestures are likely to be identified with ease using the smartwatch sensors, it is not clear how much of user's finger gestures can be recognized. In this paper, we show that motion energy measured at the smartwatch is sufficient to uniquely identify user's hand and finger gestures. We identify essential features of accelerometer and gyroscope data that reflect the movements of tendons (passing through the wrist) when performing a finger or a hand gesture. With these features, we build a classifier that can uniquely identify 37 (13 finger, 14 hand and 10 arm) gestures with an accuracy of 98 . We further extend our gesture recognition to identify the characters written by the user with her index finger on a surface, and show that such finger-writing can also be accurately recognized with nearly 95 accuracy. Our presented results will enable many novel applications like remote control and finger-writing-based input to devices using smartwatch. Generating a visually appealing human motion sequence using low-dimensional control signals is a major line of study in the motion research area in computer graphics. We propose a novel approach that allows us to reconstruct full body human locomotion using a single inertial sensing device, a smartphone. Smartphones are among the most widely used devices and incorporate inertial sensors such as an accelerometer and a gyroscope. To find a mapping between a full body pose and smartphone sensor data, we perform low dimensional embedding of full body motion capture data, based on a Gaussian Process Latent Variable Model. Our system ensures temporal coherence between the reconstructed poses by using a state decomposition model for automatic phase segmentation. Finally, application of the proposed nonlinear regression algorithm finds a proper mapping between the latent space and the sensor data. Our framework effectively reconstructs plausible 3D locomotion sequences. We compare the generated animation to ground truth data obtained using a commercial motion capture system. Wearable devices, such as smartwatches, are furnished with state-of-the-art sensors that enable a range of context-aware applications. However, malicious applications can misuse these sensors, if access is left unaudited. In this paper, we demonstrate how applications that have access to motion or inertial sensor data on a modern smartwatch can recover text typed on an external QWERTY keyboard. Due to the distinct nature of the perceptible motion sensor data, earlier research efforts on emanation based keystroke inference attacks are not readily applicable in this scenario. The proposed novel attack framework characterizes wrist movements (captured by the inertial sensors of the smartwatch worn on the wrist) observed during typing, based on the relative physical position of keys and the direction of transition between pairs of keys. Eavesdropped keystroke characteristics are then matched to candidate words in a dictionary. Multiple evaluations show that our keystroke inference framework has an alarmingly high classification accuracy and word recovery rate. With the information recovered from the wrist movements perceptible by a smartwatch, we exemplify the risks associated with unaudited access to seemingly innocuous sensors (e.g., accelerometers and gyroscopes) of wearable devices. As part of our efforts towards preventing such side-channel attacks, we also develop and evaluate a novel context-aware protection framework which can be used to automatically disable (or downgrade) access to motion sensors, whenever typing activity is detected. Scholars are increasingly citing electronic “web references” which are not preserved in libraries or full text archives. WebCite is a new standard for citing web references. To “webcite” a document involves archiving the cited Web page through www.webcitation.org and citing the WebCite permalink instead of (or in addition to) the unstable live Web page. This journal has amended its “instructions for authors” accordingly, asking authors to archive cited Web pages before submitting a manuscript. Almost 200 other journals are already using the system. We discuss the rationale for WebCite, its technology, and how scholars, editors, and publishers can benefit from the service. Citing scholars initiate an archiving process of all cited Web references, ideally before they submit a manuscript. Authors of online documents and websites which are expected to be cited by others can ensure that their work is permanently available by creating an archived copy using WebCite and providing the citation information including the WebCite link on their Web document(s). Editors should ask their authors to cache all cited Web addresses (Uniform Resource Locators, or URLs) “prospectively” before submitting their manuscripts to their journal. Editors and publishers should also instruct their copyeditors to cache cited Web material if the author has not done so already. Finally, WebCite can process publisher submitted “citing articles” (submitted for example as eXtensible Markup Language [XML] documents) to automatically archive all cited Web pages shortly before or on publication. Finally, WebCite can act as a focussed crawler, caching retrospectively references of already published articles. Copyright issues are addressed by honouring respective Internet standards (robot exclusion files, no-cache and no-archive tags). Long-term preservation is ensured by agreements with libraries and digital preservation organizations. The resulting WebCite Index may also have applications for research assessment exercises, being able to measure the impact of Web services and published Web documents through access and Web citation metrics. @PARASPLIT [J Med Internet Res 2005;7(5):e60] This paper presents an energy-aware method for recognizing time series acceleration data containing both activities and gestures using a wearable device coupled with a smartphone. In our method, we use a small wearable device to collect accelerometer data from a user's wrist, recognizing each data segment using a minimal feature set chosen automatically for that segment. For each collected data segment, if our model finds that recognizing the segment requires high-cost features that the wearable device cannot extract, such as dynamic time warping for gesture recognition, then the segment is transmitted to the smartphone where the high-cost features are extracted and recognition is performed. Otherwise, only the minimum required set of low-cost features are extracted from the segment on the wearable device and only the recognition result, i.e., label, is transmitted to the smartphone in place of the raw data, reducing transmission costs. Our method automatically constructs this adaptive processing pipeline solely from training data. The use of on-body wearable sensors is widespread in several academic and industrial domains. Of great interest are their applications in ambulatory monitoring and pervasive computing systems; here, some quantitative analysis of human motion and its automatic classification are the main computational tasks to be pursued. In this paper, we discuss how human physical activity can be classified using on-body accelerometers, with a major emphasis devoted to the computational algorithms employed for this purpose. In particular, we motivate our current interest for classifiers based on Hidden Markov Models (HMMs). An example is illustrated and discussed by analysing a dataset of accelerometer time series. User identification is an essential problem for security protection and data privacy preservation of wearable devices. With proper user identification, wearable devices can adopt personalized settings for different users, automatically label the corresponding data to protect user privacy, and help prevent illegal user spoofing attacks. Current user identification solutions proposed for wearable devices either rely on dedicated devices with high cost or require user intervention which is not convenient. In this work, we leverage the bio-vibrometry to enable a novel user identification solution for wearable devices in small-scale scenarios, e.g., household scenario. Unlike existing user identification solutions, our system only uses the low-cost sensors that are already available for most wearable devices. The key idea is that, when human body is exposed to a vibration excitation, the vibration response reflects the physical characteristics of user, i.e., the mass, stiffness and damping. Meanwhile, due to users' biological diversity, such physical characteristics of different users are quite distinctive. Therefore, we can leverage the discrepancy in users' vibration responses as an identifier. Based on this idea, we propose VibID, which only uses a low-cost vibration motor and accelerometer to generate an unobtrusive vibration to users' arms and capture the corresponding responses. By examining the vibration patterns at different frequencies, VibID builds an ensemble machine learning model to recognize who is using the device. Extensive experiments are conducted on human subjects to demonstrate that our system is reliable in small- scale scenarios and robust to various confounding factors, e.g., arm position, muscle state, user mobility and wearing location. We also show that, in an uncontrolled scenario of 8 users, our system can still ensure a identification accuracy above 91 . We propose SkinWatch, a new interaction modality for wearable devices. SkinWatch provides gesture input by sensing deformation of the skin under a wearable wrist device, also known as a smart watch. A gesture command that is matched by learning data and two-dimensional linear input recognizes the gestures. The sensing part is small, thin, and stable, to accept accurate input via a user's skin. We also implement an anti-error mechanism to prevent unexpected input when the user moves or rotates his or her forearm. The whole sensor costs less than $1.50 and the sensor layer does not exceed a height of more than 3 mm in this prototype. We demonstrate sample applications with a practical task; using two-finger skin gesture input. The ubiquity of wearable and implantable devices has sparked a new set of mobile computing applications that leverage the abundant information from sensors. For many of these applications, ensuring the security of communication between legitimate devices is a crucial problem. In this paper, we design Walkie-Talkie, a shared secret key generation scheme that allows two legitimate devices to establish a common cryptographic key by exploiting users' walking characteristics (gait). The intuition is that the sensors on different locations on the same body experience similar accelerometer signals when the user is walking. However, the accelerometer also captures motion signals produced by other body parts (e.g., swinging arms). It is shown that a Blind Source Separation (BSS) technique can extract the informative signal produced by the unique gait patterns. Our experimental results show that the keys generated by two independent devices on the same body are able to achieve up to a 100 bit agreement rate. To demonstrate the feasibility, the proposed key generation scheme is implemented on modern smartphones. The evaluation results show that the proposed scheme can run in real-time on modern mobile devices and incurs low system overhead. The proliferation of wearable devices, e.g., smartwatches and activity trackers, with embedded sensors has already shown its great potential on monitoring and inferring human daily activities. This paper reveals a serious security breach of wearable devices in the context of divulging secret information (i.e., key entries) while people accessing key-based security systems. Existing methods of obtaining such secret information relies on installations of dedicated hardware (e.g., video camera or fake keypad), or training with labeled data from body sensors, which restrict use cases in practical adversary scenarios. In this work, we show that a wearable device can be exploited to discriminate mm-level distances and directions of the user's fine-grained hand movements, which enable attackers to reproduce the trajectories of the user's hand and further to recover the secret key entries. In particular, our system confirms the possibility of using embedded sensors in wearable devices, i.e., accelerometers, gyroscopes, and magnetometers, to derive the moving distance of the user's hand between consecutive key entries regardless of the pose of the hand. Our Backward PIN-Sequence Inference algorithm exploits the inherent physical constraints between key entries to infer the complete user key entry sequence. Extensive experiments are conducted with over 5000 key entry traces collected from 20 adults for key-based security systems (i.e. ATM keypads and regular keyboards) through testing on different kinds of wearables. Results demonstrate that such a technique can achieve 80 accuracy with only one try and more than 90 accuracy with three tries, which to our knowledge, is the first technique that reveals personal PINs leveraging wearable devices without the need for labeled training data and contextual information. Given the increasing popularity of wearable devices, this paper explores the potential to use wearables for steering and driver tracking. Such capability would enable novel classes of mobile safety applications without relying on information or sensors in the vehicle. In particular, we study how wrist-mounted inertial sensors, such as those in smart watches and fitness trackers, can track steering wheel usage and angle. In particular, tracking steering wheel usage and turning angle provide fundamental techniques to improve driving detection, enhance vehicle motion tracking by mobile devices and help identify unsafe driving. The approach relies on motion features that allow distinguishing steering from other confounding hand movements. Once steering wheel usage is detected, it further uses wrist rotation measurements to infer steering wheel turning angles. Our on-road experiments show that the technique is 99 accurate in detecting steering wheel usage and can estimate turning angles with an average error within 3.4 degrees. The increased usage of smart wearables in various applications, specifically in health-care, emphasizes the need for secure communication to transmit sensitive health-data. In a practical scenario, where multiple devices are carried by a person, a common secret key is essential for secure group communication. Group key generation and sharing among wearables have received very little attention in the literature due to the underlying challenges: 1) difficulty in obtaining a good source of randomness to generate strong cryptographic keys, and 2) finding a common feature among all the devices to share the key. In this paper, we present a novel solution to generate and distribute group secret keys by exploiting on-board accelerometer sensor and the unique walking style of the user, i.e., gait. We propose a method to identify the suitable samples of accelerometer data during all routine activities of a subject to generate the keys with high entropy. In our scheme, the smartphone placed on waist employs fuzzy vault, a cryptographic construct, and utilizes the acceleration due to gait, a common characteristic extracted on all wearable devices to share the secret key. We implement our solution on commercially available off-the-shelf smart wearables, measure the system performance, and conduct experiments with multiple subjects. Our results demonstrate that the proposed solution has a bit rate of 750 b s, low system overhead, distributes the key securely and quickly to all legitimate devices, and is suitable for practical applications. To use a wrist-mounted sensor (such as a camera) for a robot task, the position and orientation of the sensor with respect to the robot wrist frame must be known. The sensor mounting position can be found by moving the robot and observing the resulting motion of the sensor. This yields a homogeneous transform equation of the form AX=XB, where A is the change in the robot wrist position, B is the resulting sensor displacement, and X is the sensor position relative to the robot wrist. The solution to an equation of this form has one degree of rotational freedom and one degree of translation freedom if the angle of rotation of A is neither 0 nor pi radians. To solve for X uniquely, it is necessary to make two arm movements and form a system of two equations of the form: A sub 1 X=XB sub 1 and A sub 2 X=XB sub 2 . A closed-form solution to this system of equations is developed and the necessary conditions for uniqueness are stated. > In this paper we propose a hand gesture recognition system that relies on color and depth images, and on a small pose sensor on the human palm. Monocular and stereo vision systems have been used for human pose and gesture recognition, but with limited scope due to limitations on texture, illumination, etc. New RGB-Depth sensors, that reply on projected light such as the Microsoft Kinect, have overcome many of those limitations. However, the point clouds for hand gestures are still in many cases noisy and partially occluded, and hand gesture recognition is not trivial. Hand gesture recognition is much more complex than full body motion, since we can have the hands in any orientation and can not assume a standing body on a ground plane. In this work we propose to add a tiny pose sensor to the human palm, with a minute accelerometer and magnetometer that combined provide 3D angular pose, to reduce the search space and have a robust and computationally light recognition method. Starting with the full depth image point cloud, segmentation can be performed by taking into account the relative depth and hand orientation, as well as skin color. Identification is then performed by matching 3D voxel occupancy against a gesture template database. Preliminary results are presented for the recognition of Portuguese Sign Language alphabet, showing the validity of the approach. The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available. Human-centered computing is an emerging research field that aims to understand human behavior and integrate users and their social context with computer systems. One of the most recent, challenging and appealing applications in this framework consists in sensing human body motion using smartphones to gather context information about people actions. In this context, we describe in this work an Activity Recognition database, built from the recordings of 30 subjects doing Activities of Daily Living (ADL) while carrying a waist-mounted smartphone with embedded inertial sensors, which is released to public domain on a well-known on-line repository. Results, obtained on the dataset by exploiting a multiclass Support Vector Machine (SVM), are also acknowledged. Previous work on muscle activity sensing has leveraged specialized sensors such as electromyography and force sensitive resistors. While these sensors show great potential for detecting finger hand gestures, they require additional hardware that adds to the cost and user discomfort. Past research has utilized sensors on commercial devices, focusing on recognizing gross hand gestures. In this work we present Serendipity, a new technique for recognizing unremarkable and fine-motor finger gestures using integrated motion sensors (accelerometer and gyroscope) in off-the-shelf smartwatches. Our system demonstrates the potential to distinguish 5 fine-motor gestures like pinching, tapping and rubbing fingers with an average f1-score of 87 . Our work is the first to explore the feasibility of using solely motion sensors on everyday wearable devices to detect fine-grained gestures. This promising technology can be deployed today on current smartwatches and has the potential to be applied to cross-device interactions, or as a tool for research in fields involving finger and hand motion. Walking speed is widely used to study human health status. Wearable inertial measurement units (IMU) are promising tools for the ambulatory measurement of walking speed. Among wearable inertial sensors, the ones worn on the wrist, such as a watch or band, have relatively higher potential to be easily incorporated into daily lifestyle. Using the arm swing motion in walking, this paper proposes a regression model-based method for longitudinal walking speed estimation using a wrist-worn IMU. A novel kinematic variable is proposed, which finds the wrist acceleration in the principal axis (i.e. the direction of the arm swing). This variable (called pca-acc) is obtained by applying sensor fusion on IMU data to find the orientation followed by the use of principal component analysis. An experimental evaluation was performed on 15 healthy young subjects during free walking trials. The experimental results show that the use of the proposed pca-acc variable can significantly improve the walking speed estimation accuracy when compared to the use of raw acceleration information (p<0.01). When Gaussian process regression is used, the resulting walking speed estimation accuracy and precision is about 5.9 and 4.7 , respectively. A citing sentence is one that appears in a scientific article and cites previous work. Citing sentences have been studied and used in many applications. For example, they have been used in scientific paper summarization, automatic survey generation, paraphrase identification, and citation function classification. Citing sentences that cite multiple papers are common in scientific writing. This observation should be taken into consideration when using citing sentences in applications. For instance, when a citing sentence is used in a summary of a scientific paper, only the fragments of the sentence that are relevant to the summarized paper should be included in the summary. In this paper, we present and compare three different approaches for identifying the fragments of a citing sentence that are related to a given target reference. Our methods are: word classification, sequence labeling, and segment classification. Our experiments show that segment classification achieves the best results. Computer vision and inertial measurement have made it possible for people to interact with computers using whole-body gestures. Although there has been rapid growth in the uses and applications of these systems, their ubiquity has been limited by the high cost of heavily instrumenting either the environment or the user. In this paper, we use the human body as an antenna for sensing whole-body gestures. Such an approach requires no instrumentation to the environment, and only minimal instrumentation to the user, and thus enables truly mobile applications. We show robust gesture recognition with an average accuracy of 93 across 12 whole-body gestures, and promising results for robust location classification within a building. In addition, we demonstrate a real-time interactive system which allows a user to interact with a computer using whole-body gestures