Over the last four years we have operated a public smartphone platform testbed called PHONELAB. PHONELAB consists of up to several-hundred participants who run an experimental platform image on their primary smartphone. The experimental platform consists of both instrumentation and experimental changes to platform components, including core Android services and Linux. This paper describes the design of the testbed, the process of conducting PHONELAB experiments, and some of the research the testbed has supported. We also offer many lessons learned along the way, almost all of which have been learned the hard way--through trial and a lot of error. We expect our experiences will help those contemplating operating large user-facing testbeds, anyone conducting experiments on smartphones, and many mobile systems researchers.
Finally, SmartLab @cite_9 is a smartphone testbed consisting of 40 Android smartphones. The smartphones are connected to a hub via USB and user interactions are simulated through a web-based remote screen terminal. The devices are neither mobile nor used by real users. provides a level of realism that SmartLab lacks.
@cite_9
2407562357 1994718979 2089790844 1973831058
SmartLab is a first-of-a-kind open cloud of smartphones that enables a new line of systems-oriented mobile computing research. We present LiveLabs, a mobile experimentation testbed that is currently deployed across our university campus with further deployments at a large shopping mall, a commercial airport, and a resort island soon to follow. The key goal of LiveLabs is to allow in-situ real-time experimentation of mobile applications and services that require context-specific triggers with real participants on their actual smart phones. We describe how LiveLabs works, and then explain the novel R&D required to realise it. We end with a description of the current LiveLabs status (> 700 active participants to date) as well as present some key lessons learned. We believe that, for successful adoption of novel mobile technologies and applications, it is important to be able to test them under real usage patterns, and with real users. To implement this vision, we present our initial effort in building LiveLabs, a large-scale mobile testbed for in-situ experimentation. LiveLabs is unique in two aspects. First, LiveLabs operates on a scale much larger than most research testbeds it is being deployed in four different public spaces in Singapore (a university campus, a shopping mall, an airport and a leisure resort), and is expected to have a pool of over 30,000 opt-in participants. Second, LiveLabs not only instruments smartphones and the infrastructure to gather deep individual and collective context, but also provides a unique experimentation platform that automates many aspects of behavioral experimentation, such as subject selection and context-triggered delivery of interventions. We briefly describe some of the research challenges associated with building such a large-scale deep-context collection testbed, as well as the current status of LiveLabs. We then share our perspectives on the challenges of setting up and operating such testbeds, with the expectation that our experiences will prove useful to other researchers looking to build similar testbeds elsewhere. Today's smartphones provide services and uses that required a panoply of dedicated devices not so long ago. With them, we listen to music, play games or chat with our friends; but we also read our corporate email and documents, manage our online banking; and we have started to use them directly as a means of payment. In this paper, we aim to raise awareness of side-channel attacks even when strong isolation protects sensitive applications. Previous works have studied the use of the phone accelerometer and gyroscope as side channel data to infer PINs. Here, we describe a new side-channel attack that makes use of the video camera and microphone to infer PINs entered on a number-only soft keyboard on a smartphone. The microphone is used to detect touch events, while the camera is used to estimate the smartphone's orientation, and correlate it to the position of the digit tapped by the user. We present the design, implementation and early evaluation of PIN Skimmer, which has a mobile application and a server component. The mobile application collects touch-event orientation patterns and later uses learnt patterns to infer PINs entered in a sensitive application. When selecting from a test set of 50 4-digit PINs, PIN Skimmer correctly infers more than 30 of PINs after 2 attempts, and more than 50 of PINs after 5 attempts on android-powered Nexus S and Galaxy S3 phones. When selecting from a set of 200 8-digit PINs, PIN Skimmer correctly infers about 45 of the PINs after 5 attempts and 60 after 10 attempts. It turns out to be difficult to prevent such side-channel attacks, so we provide guidelines for developers to mitigate present and future side-channel attacks on PIN input.