We introduce Pixie, a novel, camera based two factor authentication solution for mobile and wearable devices. A quick and familiar user action of snapping a photo is sufficient for Pixie to simultaneously perform a graphical password authentication and a physical token based authentication, yet it does not require any expensive, uncommon hardware. Pixie establishes trust based on both the knowledge and possession of an arbitrary physical object readily accessible to the user, called trinket. Users choose their trinkets similar to setting a password, and authenticate by presenting the same trinket to the camera. The fact that the object is the trinket, is secret to the user. Pixie extracts robust, novel features from trinket images, and leverages a supervised learning classifier to effectively address inconsistencies between images of the same trinket captured in different circumstances. Pixie achieved a false accept rate below 0.09 in a brute force attack with 14.3 million authentication attempts, generated with 40,000 trinket images that we captured and collected from public datasets. We identify master images, that match multiple trinkets, and study techniques to reduce their impact. In a user study with 42 participants over 8 days in 3 sessions we found that Pixie outperforms text based passwords on memorability, speed, and user preference. Furthermore, Pixie was easily discoverable by new users and accurate under field use. Users were able to remember their trinkets 2 and 7 days after registering them, without any practice between the 3 test dates.
Similar to Pixie, cameras integrated in wearable devices have been used to capture the input for authentication. Van @cite_49 exploit the smartwatch camera to provide the device with an input (e.g. PIN) that is drawn on a canvas, then use image processing techniques to interpret the captured input. @cite_62 propose to pair and unlock smartglasses with the user smartphone by exploiting the glass camera to scan a QR code that is displayed on the user's phone screen. Similarly, @cite_39 use the smartglass camera to scan a QR code that is displayed on a point-of-service terminals (e.g. ATM) to connect to a cloud server for obtaining an OTP.
@cite_62 @cite_49 @cite_39
2294497986 2025636515 1586237437
Wearable computing devices have become increasingly popular and while these devices promise to improve our lives, they come with new challenges. This paper focuses on user authentication mechanisms for the Google Glass device (Glass). Glass only has three sources of input: a camera, a microphone, and a touchpad. This limited set of interfaces makes the use of standard passwords infeasible or cumbersome. We therefore propose a One-Time-Password (OTP) authentication scheme, Glass OTP that uses the Glass camera to scan a QR code displayed on the user’s smartphone. We implement a proof of concept Glass lock screen which unlocks only upon scanning an OTP generated by the companion Android smartphone application. We also discuss the reliability, security, and convenience of the proposed solution as compared to the current solutions in use. Credit card fraud is a common problem in today's world. Financial institutions have registered major loses till today due to users being exposed of their credit card information. Shoulder-surfing or observation attacks, including card skimming and video recording with hidden cameras while users perform PIN-based authentication at ATM terminals is one of the common threats for common users. Researchers have struggled to come up with secure solutions for secure PIN authentication. However, modern day ubiquitous wearable devices, such as the Google Glass have presented us with newer opportunities in this research area. In this paper, we propose Secure-PIN-Authentication-as-a-Service (SEPIA), a secure obfuscated PIN authentication protocol for ATM and other point-of-service terminals using cloud-connected personal mobile and wearable devices. Our approach protects the user from shoulder-surfers and partial observation attacks, and is also resistant to relay, replay, and intermediate transaction attacks. A SEPIA user utilizes a Google Glass or a mobile device for scanning a QR code on the terminal screen to prove co-location to the cloud-based server and obtain a secure PIN template for point-of-service authentication. SEPIA ensures minimal task overhead on the user's device with maximal computation offloaded to the cloud. We have implemented a proof-of-concept prototype to perform experimental analysis and a usability study for the SEPIA architecture. This paper shows new algorithms and the implementations of image reorganization for EAN QR barcodes in mobile phones. The mobile phone system used here consists of a camera, mobile application processor, digital signal processor (DSP), and display device, and the source image is captured by the embedded camera device. The introduced algorithm is based on the code area found by four corners detection for 2D barcode and spiral scanning for 1D barcode using the embedded DSP. This algorithm is robust for practical situations and the DSP has good enough performance for the real-time recognition of the codes. The performance of our image processing is 66.7 frames sec for EAN code and 14.1 frames sec for QR code image processing, and this is sufficient performance for practical use. The released mobile phone had performance of 5-10 frames sec including OS and subsystem overheads. This paper presents Glass Unlock, a novel concept using smart glasses for smartphone unlocking, which is theoretically secure against smudge attacks, shoulder-surfing, and camera attacks. By introducing an additional temporary secret like the layout of digits that is only shown on the private near-eye display, attackers cannot make sense of the observed input on the almost empty phone screen. We report a user study with three alternative input methods and compare them to current state-of-the-art systems. Our findings show that Glass Unlock only moderately increases authentication times and that users favor the input method yielding the slowest input times as it avoids focus switches between displays. Smartwatches can facilitate several tasks that are performed on mobile devices. However, due to their limited size, touch interaction can be cumbersome. Although alternative input modalities such as speech input could be used, these can also introduce other issues regarding privacy or ease of use. Consequently, HCI researchers have been exploring novel input techniques for smartwatches. In this paper, we introduce WatchMe, a smartwatch application that uses the smartwatch camera and image processing techniques to allow for providing input on a drawing canvas composed of everyday objects and surfaces. We rely on a cloud OCR engine to retrieve text from captured images. Combining these characteristics, we illustrate some scenarios in which WatchMe could be used, such as a novel method for two-factor authentication. This paper proposes a challenge-response authentication system for web applications called Snap2Pass that is easy to use, provides strong security guarantees, and requires no browser extensions. The system uses QR codes which are small two-dimensional pictures that encode digital data. When logging in to a site, the web server sends the PC browser a QR code that encodes a cryptographic challenge; the user takes a picture of the QR code with his cell phone camera which results in a cryptographic response sent to the server; the web server then logs the PC browser in. Our user study shows that authentication using Snap2Pass is easy to learn and considerably faster than existing one-time password and challenge-response systems. By implementing our solution as an OpenID provider, we have made this scheme available to over 30,000 websites that use OpenID today. This paper also proposes Snap2Pay, an extension of Snap2Pass, to improve the usability and security of online payments. Snap2Pay allows a consumer to use one-time credit cards as well as the Verified by Visa or Mastercard SecureCode services securely and easily with just a snap of a QR code. Traditional user authentication methods using passcode or finger movement on smartphones are vulnerable to shoulder surfing attack, smudge attack, and keylogger attack. These attacks are able to infer a passcode based on the information collection of user’s finger movement or tapping input. As an alternative user authentication approach, eye tracking can reduce the risk of suffering those attacks effectively because no hand input is required. However, most existing eye tracking techniques are designed for large screen devices. Many of them depend on special hardware like high resolution eye tracker and special process like calibration, which are not readily available for smartphone users. In this paper, we propose a new eye tracking method for user authentication on a smartphone. It utilizes the smartphone’s front camera to capture a user’s eye movement trajectories which are used as the input of user authentication. No special hardware or calibration process is needed. We develop a prototype and evaluate its effectiveness on an Android smartphone. We recruit a group of volunteers to participate in the user study. Our evaluation results show that the proposed eye tracking technique achieves very high accuracy in user authentication. Easily establishing pairing between Internet-of-Things (IoT) devices is important for fast deployment in many smart home scenarios. Traditional pairing methods, including passkey, QR code, and RFID, often require specific user interfaces, surface's shape material, or additional tags readers. The growing number of low-resource IoT devices without an interface may not meet these requirements, which makes their pairing a challenge. On the other hand, these devices often already have sensors embedded for sensing tasks, such as inertial sensors. These sensors can be used for limited user interaction with the devices, but are not suitable for pairing on their own. In this paper, we present UniverSense, an alternative pairing method between low-resource IoT devices with an inertial sensor and a more powerful networked device equipped with a camera. To establish pairing between them, the user moves the low-resource IoT device in front of the camera. Both the camera and the on-device sensors capture the physical motion of the low-resource device. UniverSense converts these signals into a common state-space to generate fingerprints for pairing. We conduct real-world experiments to evaluate UniverSense and it achieves an F1 score of 99.9 in experiments carried out by five participants. The proliferation of wearable devices, e.g., smartwatches and activity trackers, with embedded sensors has already shown its great potential on monitoring and inferring human daily activities. This paper reveals a serious security breach of wearable devices in the context of divulging secret information (i.e., key entries) while people accessing key-based security systems. Existing methods of obtaining such secret information relies on installations of dedicated hardware (e.g., video camera or fake keypad), or training with labeled data from body sensors, which restrict use cases in practical adversary scenarios. In this work, we show that a wearable device can be exploited to discriminate mm-level distances and directions of the user's fine-grained hand movements, which enable attackers to reproduce the trajectories of the user's hand and further to recover the secret key entries. In particular, our system confirms the possibility of using embedded sensors in wearable devices, i.e., accelerometers, gyroscopes, and magnetometers, to derive the moving distance of the user's hand between consecutive key entries regardless of the pose of the hand. Our Backward PIN-Sequence Inference algorithm exploits the inherent physical constraints between key entries to infer the complete user key entry sequence. Extensive experiments are conducted with over 5000 key entry traces collected from 20 adults for key-based security systems (i.e. ATM keypads and regular keyboards) through testing on different kinds of wearables. Results demonstrate that such a technique can achieve 80 accuracy with only one try and more than 90 accuracy with three tries, which to our knowledge, is the first technique that reveals personal PINs leveraging wearable devices without the need for labeled training data and contextual information. Wearable devices are computationally rich, portable and wearable to suit users in various needs and situations. Still, users of wearable devices suffer from lack of efficient input methods and proper feedback modality. In this paper, we design and implement a novel input method exploiting readily available ambient light sensor on wearable devices. We present concepts and implementation results of re-purposing the light sensor for user input and processing UI events. Furthermore, a PIN entry application based on the proposed input method is implemented to demonstrate a device authentication scenario for wearables. To prevent unauthorized parties from accessing data stored on their smartphones, users have the option of enabling a "lock screen" that requires a secret code (e.g., PIN, drawing a pattern, or biometric) to gain access to their devices. We present a detailed analysis of the smartphone locking mechanisms currently available to billions of smartphone users worldwide. Through a month-long field study, we logged events from a panel of users with instrumented smartphones (N=134). We are able to show how existing lock screen mechanisms provide users with distinct tradeoffs between usability (unlocking speed vs. unlocking frequency) and security. We find that PIN users take longer to enter their codes, but commit fewer errors than pattern users, who unlock more frequently and are very prone to errors. Overall, PIN and pattern users spent the same amount of time unlocking their devices on average. Additionally, unlock performance seemed unaffected for users enabling the stealth mode for patterns. Based on our results, we identify areas where device locking mechanisms can be improved to result in fewer human errors -- increasing usability -- while also maintaining security. The use of tablet PCs is spreading rapidly, and accordingly users browsing and inputting personal information in public spaces can often be seen by third parties. Unlike conventional mobile phones and notebook PCs equipped with distinct input devices (e.g., keyboards), tablet PCs have touchscreen keyboards for data input. Such integration of display and input device increases the potential for harm when the display is captured by malicious attackers. This paper presents the description of reconstructing tablet PC displays via measurement of electromagnetic (EM) emanation. In conventional studies, such EM display capture has been achieved by using non-portable setups. Those studies also assumed that a large amount of time was available in advance of capture to obtain the electrical parameters of the target display. In contrast, this paper demonstrates that such EM display capture is feasible in real time by a setup that fits in an attache case. The screen image reconstruction is achieved by performing a prior course profiling and a complemental signal processing instead of the conventional fine parameter tuning. Such complemental processing can eliminate the differences of leakage parameters among individuals and therefore correct the distortions of images. The attack distance, 2 m, makes this method a practical threat to general tablet PCs in public places. This paper discusses possible attack scenarios based on the setup described above. In addition, we describe a mechanism of EM emanation from tablet PCs and a countermeasure against such EM display capture. Touch-enabled user interfaces have become ubiquitous, such as on ATMs or portable devices. At the same time, authentication using touch input is problematic, since finger smudge traces may allow attackers to reconstruct passwords. We present SmudgeSafe, an authentication system that uses random geometric image transformations, such as translation, rotation, scaling, shearing, and flipping, to increase the security of cued-recall graphical passwords. We describe the design space of these transformations and report on two user studies: A lab-based security study involving 20 participants in attacking user-defined passwords, using high quality pictures of real smudge traces captured on a mobile phone display; and an in-the-field usability study with 374 participants who generated more than 130,000 logins on a mobile phone implementation of SmudgeSafe. Results show that SmudgeSafe significantly increases security compared to authentication schemes based on PINs and lock patterns, and exhibits very high learnability, efficiency, and memorability. User interaction is an essential feature in the design of an interactive game. Most existing games receive inputs from users via conventional devices such as keyboard, mouse, joystick and paddle. More recent games make use of infrared beams from user's devices, the stylus from touch screen, or pressure-sensing pads to provide rich contextual sensing and interactions. In this paper, we propose the use of hand gestures as the basis for users to directly interact with game objects that are rendered across a flat plasma or LCD display. It forms a new paradigm of interaction in which the physical movements of hands in the form of hand gestures are coordinated along with the virtual objects in the game. Thus, the user effectively becomes an “input device”. We make use of a low-cost web camera that is mounted over the gaming screen display to provide image-feed to the hand tracking and gesture recognition system, called Germane, which employs the hull-point analysis algorithm for gesture recognition. A working prototype of Germane has been developed to validate its operations on several common gestures. Performance evaluation results of Germane are also presented. PINs and patterns remain among the most widely used knowledge-based authentication schemes. As thermal cameras become ubiquitous and affordable, we foresee a new form of threat to user privacy on mobile devices. Thermal cameras allow performing thermal attacks, where heat traces, resulting from authentication, can be used to reconstruct passwords. In this work we investigate in details the viability of exploiting thermal imaging to infer PINs and patterns on mobile devices. We present a study (N=18) where we evaluated how properties of PINs and patterns influence their thermal attacks resistance. We found that thermal attacks are indeed viable on mobile devices; overlapping patterns significantly decrease successful thermal attack rate from 100 to 16.67 , while PINs remain vulnerable (>72 success rate) even with duplicate digits. We conclude by recommendations for users and designers of authentication schemes on how to resist thermal attacks. Today's smartphones provide services and uses that required a panoply of dedicated devices not so long ago. With them, we listen to music, play games or chat with our friends; but we also read our corporate email and documents, manage our online banking; and we have started to use them directly as a means of payment. In this paper, we aim to raise awareness of side-channel attacks even when strong isolation protects sensitive applications. Previous works have studied the use of the phone accelerometer and gyroscope as side channel data to infer PINs. Here, we describe a new side-channel attack that makes use of the video camera and microphone to infer PINs entered on a number-only soft keyboard on a smartphone. The microphone is used to detect touch events, while the camera is used to estimate the smartphone's orientation, and correlate it to the position of the digit tapped by the user. We present the design, implementation and early evaluation of PIN Skimmer, which has a mobile application and a server component. The mobile application collects touch-event orientation patterns and later uses learnt patterns to infer PINs entered in a sensitive application. When selecting from a test set of 50 4-digit PINs, PIN Skimmer correctly infers more than 30 of PINs after 2 attempts, and more than 50 of PINs after 5 attempts on android-powered Nexus S and Galaxy S3 phones. When selecting from a set of 200 8-digit PINs, PIN Skimmer correctly infers about 45 of the PINs after 5 attempts and 60 after 10 attempts. It turns out to be difficult to prevent such side-channel attacks, so we provide guidelines for developers to mitigate present and future side-channel attacks on PIN input. With the rich functionalities and enhanced computing capabilities available on mobile computing devices with touch screens, users not only store sensitive information (such as credit card numbers) but also use privacy sensitive applications (such as online banking) on these devices, which make them hot targets for hackers and thieves. To protect private information, such devices typically lock themselves after a few minutes of inactivity and prompt a password PIN pattern screen when reactivated. Passwords PINs patterns based schemes are inherently vulnerable to shoulder surfing attacks and smudge attacks. Furthermore, passwords PINs patterns are inconvenient for users to enter frequently. In this paper, we propose GEAT, a gesture based user authentication scheme for the secure unlocking of touch screen devices. Unlike existing authentication schemes for touch screen devices, which use what user inputs as the authentication secret, GEAT authenticates users mainly based on how they input, using distinguishing features such as finger velocity, device acceleration, and stroke time. Even if attackers see what gesture a user performs, they cannot reproduce the behavior of the user doing gestures through shoulder surfing or smudge attacks. We implemented GEAT on Samsung Focus running Windows, collected 15009 gesture samples from 50 volunteers, and conducted real-world experiments to evaluate GEAT's performance. Experimental results show that our scheme achieves an average equal error rate of 0.5 with 3 gestures using only 25 training samples. In this paper we present zSense, which provides greater input expressivity for spatially limited devices such as smart wearables through a shallow depth gesture recognition system using non-focused infrared sensors. To achieve this, we introduce a novel Non-linear Spatial Sampling (NSS) technique that significantly cuts down the number of required infrared sensors and emitters. These can be arranged in many different configurations; for example, number of sensor emitter units can be as minimal as one sensor and two emitters. We implemented different configurations of zSense on smart wearables such as smartwatches, smartglasses and smart rings. These configurations naturally fit into the flat or curved surfaces of such devices, providing a wide scope of zSense enabled application scenarios. Our evaluations reported over 94.8 gesture recognition accuracy across all configurations. Due to the rapid proliferation of image capturing devices and user-friendly editing software suites, image manipulation is at everyone's hand. For this reason, the forensic community has developed a series of techniques to determine image authenticity. In this paper, we propose an algorithm for image tampering detection and localization, leveraging characteristic footprints left on images by different camera models. The rationale behind our algorithm is that all pixels of pristine images should be detected as being shot with a single device. Conversely, if a picture is obtained through image composition, traces of multiple devices can be detected. The proposed algorithm exploits a convolutional neural network (CNN) to extract characteristic camera model features from image patches. These features are then analyzed by means of iterative clustering techniques in order to detect whether an image has been forged, and localize the alien region. One rising trend in today's consumer electronics is the wearable devices, e.g., smartwatches. With tens of millions of smartwatches shipped, however, the security implications of such devices are not fully understood. Although previous studies have pointed out some privacy concerns about the data that can be collected, like personalized health information, the threat is considered low as the leaked data is not highly sensitive and there is no real attack implemented. In this paper we investigate a security problem coming from sensors in smartwatches, especially the accelerometer. The results show that the actual threat is much beyond people's awareness. Being worn on the wrist, the accelerometer built within a smartwatch can track user's hand movements, which makes inferring user inputs on keyboards possible in theory. But several challenges need to be addressed ahead in the real-world settings: e.g., small and irregular hand movements occur persistently during typing, which degrades the tracking accuracy and sometimes even overwhelms useful signals. In this paper, we present a new and practical side-channel attack to infer user inputs on keyboards by exploiting sensors in smartwatch. Novel keystroke inference models are developed to mitigate the negative impacts of tracking noises. We focus on two major categories of keyboards: one is numeric keypad that is generally used to input digits, and the other is QWERTY keyboard on which a user can type English text. Two prototypes have been built to infer users' banking PINs and English text when they type on POS terminal and QWERTY keyboard respectively. Our results show that for numeric keyboard, the probability of finding banking PINs in the top 3 candidates can reach 65 , while for QWERTY keyboard, a significant accuracy improvement is achieved compared to the previous works, especially of the success rate of finding the correct word in the top 10 candidates.