Human errors in performing security-critical tasks are typically blamed on the complexity of those tasks. However, such errors can also occur because of (possibly unexpected) sensory distractions. A sensory distraction that produces negative effects can be abused by the adversary that controls the environment. Meanwhile, a distraction with positive effects can be artificially introduced to improve user performance. The goal of this work is to explore the effects of visual stimuli on the performance of security-critical tasks. To this end, we experimented with a large number of subjects who were exposed to a range of unexpected visual stimuli while attempting to perform Bluetooth Pairing. Our results clearly demonstrate substantially increased task completion times and markedly lower task success rates. These negative effects are noteworthy, especially, when contrasted with prior results on audio distractions which had positive effects on performance of similar tasks. Experiments were conducted in a novel (fully automated and completely unattended) experimental environment. This yielded more uniform experiments, better scalability and significantly lower financial and logistical burdens. We discuss this experience, including benefits and limitations of the unattended automated experiment paradigm.
Other than recent results describing effects of audio distractions @cite_13 , we are unaware of any prior usability studies utilizing a fully automated and unattended physical environment.
@cite_13
1830858207 1685619946 2769512728 2394679689 2009127721 2144533308 2108951007 2095416579 2154998637 2133658999 2018946706 2028086564 2012895915 2343672207 2029199293 2118847468 2540183746 2169074609 2048020986 24931679
Author(s): Kaczmarek, T; Kobsa, A; Sy, R; Tsudik, G | Abstract: User errors while performing security-critical tasks can lead to undesirable or even disastrous consequences. One major factor influencing mistakes and failures is complexity of such tasks, which has been studied extensively in prior research. Another important issue which hardly received any attention is the impact of both accidental and intended distractions on users performing security-critical tasks. In particular, it is unclear whether, and to what extent, unexpected sensory cues (e.g., auditory or visual) can influence user behavior and or trigger mistakes. Better understanding of the effects of intended distractions will help clarify their role in adversarial models. As part of the research effort described in this paper, we administered a range of naturally occurring -- yet unexpected -- sounds while study participants attempted to perform a security-critical task. We found that, although these auditory cues lowered participants' failure rates, they had no discernible effect on their task completion times. To this end, we overview some relevant literature that explains these somewhat counter-intuitive findings. Conducting a thorough and meaningful study on user errors requires a large number of participants, since errors are typically infrequent and should not be instigated more than once per subject. To reduce the effort of running numerous subjects, we developed a novel experimental setup that was fully automated and unattended. We discuss our experience with this setup and highlight the pros and cons of generalizing its usage. It is well documented that interacting with a mobile phone is associated with poorer performance on concurrently performed tasks because limited attentional resources must be shared between tasks. However, mobile phones generate auditory or tactile notifications to alert users of incoming calls and messages. Although these notifications are generally short in duration, they can prompt task-irrelevant thoughts, or mind wandering, which has been shown to damage task performance. We found that cellular phone notifications alone significantly disrupted performance on an attention-demanding task, even when participants did not directly interact with a mobile device during the task. The magnitude of observed distraction effects was comparable in magnitude to those seen when users actively used a mobile phone, either for voice calls or text messaging. (PsycINFO Database Record Keywords: Driver distraction; Language: en Ambient sensory distractions play a role in security-critical tasks that require human involvement. Adversarial control over human sensory input could broaden the attack surface. To shed light on this issue, a team conducted large-scale experiments that exposed subjects to unexpected audio and visual stimuli while they performed a security-critical task. As smartphones increasingly pervade our daily lives, people are ever more interrupted by alerts and notifications. Using both correlational and experimental methods, we explored whether such interruptions might be causing inattention and hyperactivity-symptoms associated with Attention Deficit Hyperactivity Disorder (ADHD) even in people not clinically diagnosed with ADHD. We recruited a sample of 221 participants from the general population. For one week, participants were assigned to maximize phone interruptions by keeping notification alerts on and their phones within their reach sight. During another week, participants were assigned to minimize phone interruptions by keeping alerts off and their phones away. Participants reported higher levels of inattention and hyperactivity when alerts were on than when alerts were off. Higher levels of inattention in turn predicted lower productivity and psychological well-being. These findings highlight some of the costs of ubiquitous connectivity and suggest how people can reduce these costs simply by adjusting existing phone settings. This paper examines the hypothesis that information flow, from device to user, is a vital part of skilled activity in human-computer interaction. Two studies are reported. The first study questions users of keyboard-driven word processors about the effects of cursor-movement, finding and word-deletion commands in various contexts. The second study questions users of the Apple Macintosh-based systems, MacWrite and Microsoft Word, about the behaviour of the menu-driven find command. In both studies it is discovered that users often do not know the precise effects of frequently-used actions, such as the final position of the cursor, even though these effects are vital for future planning. It is concluded that even experienced users must acquire the information they need from the device's display during interactions, and that they do not necessarily remember regular details that are available in this way. This conclusion conflicts with those current models of user psychology that assume routine skill relies on complete mental specifications of methods for performing tasks. Smartphone users might be interrupted while interacting with an application, either by intended or unintended circumstances. In this paper, we report on a large-scale observational study that investigated mobile application interruptions in two scenarios: (1) intended back and forth switching between applications and (2) unintended interruptions caused by incoming phone calls. Our findings reveal that these interruptions rarely happen (at most 10 of the daily application usage), but when they do, they may introduce a significant overhead (can delay completion of a task by up to 4 times). We conclude with a discussion of the results, their limitations, and a series of implications for the design of mobile phones. A common assumption in studies of interruptions is that one is focused in an activity and then distracted by other stimuli. We take the reverse perspective and examine whether one might first be in an attentional state that makes one susceptible to communications typically associated with distraction. We explore the confluence of multitasking and workplace communications from three temporal perspectives -- prior to an interaction, when tasks and communications are interleaved, and at the end of the day. Using logging techniques and experience sampling, we observed 32 employees in situ for five days. We found that certain attentional states lead people to be more susceptible to particular types of interaction. Rote work is followed by more Facebook or face-to-face interaction. Focused and aroused states are followed by more email. The more time in email and face-fo-face interaction, and the more total screen switches, the less productive people feel at the day's end. We present the notion of emotional homeostasis along with new directions for multitasking research. Recently, multitasking on smart phones during navigation has emerged as a problematic social behavior due to its potential danger. In this paper, we propose to alleviate the situation by employing a sensor system for obstacle detection and aid the multi-tasking user so that one can safely navigate and carry out the on-going secondary task as effectively as possible. As such, we have implemented an ultrasonic sensor system interfaced into the smart phone that can constantly appraise the user of the incoming obstacles. We ran experiments to validate our approach, checking whether such a system would help user bump less into obstacles than without, and observe their multitasking behavior such as the physical attentional switch. Our experiments have shown that with the aid of the sensor system, the user's attention switch was significantly reduced, however, there was no differences in the performance (e.g. no. of collision) because the momentary strategic spatial and path planning was quite effective with slow pace and light pedestrian traffic. We conclude therefore human dual or multitasking ability is sufficient to overlap "casual" video watching and "slow" navigation. However, with more stringent task, e.g. text messaging and heavier traffic, we expect the reduced attention shift to ultimately improve the performance as well. Problem: The prevalence of automobile drivers talking on cell phones is growing, but the effect of that behavior on driving performance is unclear. Also unclear is the relationship between the difficulty level of a phone conversation and the resulting distraction. Method: This study used a driving simulator to determine the effect that easy and difficult cell phone conversations have on driving performance. Results: Cell phone use caused participants to have higher variation in accelerator pedal position, drive more slowly with more variation in speed, and report a higher level of workload regardless of conversation difficulty level. Conclusions: Drivers may cope with the additional stress of phone conversations by enduring higher workloads or setting reduced performance goals. Impact on Industry: Because an increasing number of people talk on the phone while driving, crashes caused by distracted drivers using cell phones will cause disruptions in business, as well as injury, disability, and permanent loss of personnel. D 2004 National Safety Council and Elsevier Ltd. All rights reserved. As use of in-vehicle information systems (IVISs) such as cell phones, navigation systems, and satellite radios has increased, driver distraction has become an important and growing safety concern. A promising way to overcome this problem is to detect driver distraction and adapt in-vehicle systems accordingly to mitigate such distractions. To realize this strategy, this paper applied support vector machines (SVMs), which is a data mining method, to develop a real-time approach for detecting cognitive distraction using drivers' eye movements and driving performance data. Data were collected in a simulator experiment in which ten participants interacted with an IVIS while driving. The data were used to train and test both SVM and logistic regression models, and three different model characteristics were investigated: how distraction was defined, which data were input to the model, and how the input data were summarized. The results show that the SVM models were able to detect driver distraction with an average accuracy of 81.1 , outperforming more traditional logistic regression models. The best performing model (96.1 accuracy) resulted when distraction was defined using experimental conditions (i.e., IVIS drive or baseline drive), the input data were comprised of eye movement and driving measures, and these data were summarized over a 40-s window with 95 overlap of windows. These results demonstrate that eye movements and simple measures of driving performance can be used to detect driver distraction in real time. Potential applications of this paper include the design of adaptive in-vehicle systems and the evaluation of driver distraction A partial report procedure was used to test the ability of observers to split attention over noncontiguous locations. Observers reported the identity of 2 targets that appeared within a 5 X 5 stimulus array, and cues (validity = 80 ) informed them of the 2 most likely target locations. On invalid trials, 1 of the targets appeared directly in between the cued locations. Experiments 1, la, and 2 showed a strong accuracy advantage at cued locations compared with intervening ones. This effect was larger when the cues were arranged horizontally rather than vertically. Experiment 3 suggests that this effect of cue orientation reflects an advantage for processing targets that appear in different hemifields. Experiments 4 and 4a suggest that the primary mechanism supporting the flexible deployment of spatial attention is the suppression of interference from stimuli at unattended locations. There is a substantial body of research showing that when observers direct attention to specific parts of the visual field, information processing is facilitated at attended locations relative to unattended locations. A basic question is how flexibly attention can be deployed over space. Most models of spatial attention have assumed that attention is allocated over contiguous regions of space (e.g., Eriksen & Yeh, 1985; Posner, Snyder, & Davidson, 1980), and various empirical tests have supported this claim (e.g., Eriksen & Yeh, 1985; Heinze, Luck, Muente, Goes, Mangun, & Hillyard, 1994; McCormick & Klein, 1990; Pan & Eriksen, 1993; Posner, Snyder, & Davidson, 1980). However, more recent evidence has suggested that allocation of attention to noncontiguous regions is possible. Kramer and Hahn (1995) and Hahn and Kramer (1998) studied the influence of distractor compatibility at locations directly in between two cued locations and found that under certain conditions performance was unaffected by the distractors. In particular, they noted that when the distractors were presented by removing selected parts of a premask (the nononset condition), observers were able to suppress their influence. However, when the intervening distractors were presented as sudden-onset stimuli, distractor compatibility effects emerged. The investigators concluded that observers could divide attention effectively between noncontiguous locations but that this split focus was disrupted when new perceptual objects appeared in the intervening locations. They used a very strict criterion for evidence of split attention, however—namely, that distractors have absolutely no effect on target processing. The fact We identify usability challenges facing consumers adopting Virtual Reality (VR) head-mounted displays (HMDs) in a survey of 108 VR HMD users. Users reported significant issues in interacting with, and being aware of their real-world context when using a HMD. Building upon existing work on blending real and virtual environments, we performed three design studies to address these usability concerns. In a typing study, we show that augmenting VR with a view of reality significantly corrected the performance impairment of typing in VR. We then investigated how much reality should be incorporated and when, so as to preserve users' sense of presence in VR. For interaction with objects and peripherals, we found that selectively presenting reality as users engaged with it was optimal in terms of performance and users' sense of presence. Finally, we investigated how this selective, engagement-dependent approach could be applied in social environments, to support the user's awareness of the proximity and presence of others. We have developed a system for remedial reading instruction that uses visually controlled auditory prompting to help the user with recognition and pronunciation of words. Our underlying hypothesis is that the relatively unobtrusive assistance rendered by such a system will be more effective than previous computer aided approaches. We present a description of the design and implementation of our system and discuss a controlled study that we undertook to evaluate the usability of the Reading Assistant. Driver distraction has been identified as one major cause of unsafe driving. The existing studies on cognitive distraction detection mainly focused on high-speed driving situations, but less on low-speed traffic in urban driving. This paper presents a method for the detection of driver cognitive distraction at stop-controlled intersections and compares its feature subsets and classification accuracy with that on a speed-limited highway. In the simulator study, 27 subjects were recruited to participate. Driver cognitive distraction is induced by the clock task that taxes visuospatial working memory. The support vector machine (SVM) recursive feature elimination algorithm is used to extract an optimal feature subset out of features constructed from driving performance and eye movement. After feature extraction, the SVM classifier is trained and cross-validated within subjects. On average, the classifier based on the fusion of driving performance and eye movement yields the best correct rate and F-measure ( @math for stop-controlled intersections and @math for a speed-limited highway) among four types of the SVM model based on different candidate features. The comparisons of extracted optimal feature subsets and the SVM performance between two typical driving scenarios are presented. Automatic audio-visual speech recognition currently lags behind its audio-only counterpart in terms of major progress. One of the reasons commonly cited by researchers is the scarcity of suitable research corpora. This paper details the creation of a new corpus designed for continuous audio-visual speech recognition research . TCD-TIMIT consists of high-quality audio and video footage of 62 speakers reading a total of 6913 phonetically rich sentences. Three of the speakers are professionally-trained lipspeakers, recorded to test the hypothesis that lipspeakers may have an advantage over regular speakers in automatic visual speech recognition systems. Video footage was recorded from two angles: straight on, and at @math . The paper outlines the recording of footage, and the required post-processing to yield video and audio clips for each sentence. Audio, visual, and joint audio-visual baseline experiments are reported. Separate experiments were run on the lipspeaker and non-lipspeaker data, and the results compared. Visual and audio-visual baseline results on the non-lipspeakers were low overall. Results on the lipspeakers were found to be significantly higher. It is hoped that as a publicly available database, TCD-TIMIT will now help further state of the art in audio-visual speech recognition research. This paper reviews definitions of audio-visual synchrony and examines their empirical behaviour on test sets up to 200 times larger than used by other authors. The results give new insights into the practical utility of existing synchrony definitions and justify application of audio-visual synchrony techniques to the problem of active speaker localisation in broadcast video. Performance is evaluated using a test set of twelve clips of alternating speakers from the multiple speaker CUAVE corpus. Accuracy of 76 is obtained for the task of identifying the active member of a speaker pair at different points in time, comparable to performance given by two purely video image-based schemes. Accuracy of 65 is obtained on the more challenging task of locating a point within a 100×100 pixel square centered on the active speaker's mouth without no prior face detection; the performance upper bound if perfect face detection were available is 69 . This result is significantly better than two purely video image-based schemes. Driver distraction, defined as the diversion of attention away from activities critical for safe driving toward a competing activity, is increasingly recognized as a significant source of injuries and fatalities on the roadway. Additionally, the trend towards increasing the use of in-vehicle information systems is critical because they induce visual, biomechanical and cognitive distraction and may affect driving performance in qualitatively different ways. Non-intrusive methods are strongly preferred for monitoring distraction, and vision-based systems have appeared to be attractive for both drivers and researchers. Biomechanical, visual and cognitive distractions are the most commonly detected types in video-based algorithms. Many distraction detection systems only use a single visual cue and therefore, they may be easily disturbed when occlusion or illumination changes appear. Moreover, the combination of these visual cues is a key and challenging aspect in the development of robust distraction detection systems. These visual cues can be extracted mainly by using face monitoring systems but they should be completed with more visual cues (e.g., hands or body information) or even, distraction detection from specific actions (e.g., phone usage). Additionally, these algorithms should be included in an embedded device or system inside a car. This is not a trivial task and several requirements must be taken into account: reliability, real-time performance, low cost, small size, low power consumption, flexibility and short time-to-market. The key points for the development and implementation of sensors to carry out the detection of distraction will also be reviewed. This paper shows a review of the role of computer vision technology applied to the development of monitoring systems to detect distraction. Some key points considered as both future work and challenges ahead yet to be solved will also be addressed. In this paper we present the results of a study that examines the user's perception-understood as both information assimilation and subjective satisfaction-of multimedia quality, when impacted by varying network-level parameters (delay and jitter). In addition, we integrate eye-tracking assessment to provide a more complete understanding of user perception of multimedia quality. Results show that delay and jitter significantly affect user satisfaction; variation in video eye path when either no single obvious point of focus exists or when the point of attention changes dramatically. Lastly, results showed that content variation significantly affected user satisfaction, as well as user information assimilation. User attention is a scarce resource, and users are susceptible to interruption overload. Systems do not reason about the effects of interrupting a user during a task sequence. In this study, we measure effects of interrupting a user at different moments within task execution in terms of task performance, emotional state, and social attribution. Task models were developed using event perception techniques, and the resulting models were used to identify interruption timings based on a user's predicted cognitive load. Our results show that different interruption moments have different impacts on user emotional state and positive social attribution, and suggest that a system could enable a user to maintain a high level of awareness while mitigating the disruptive effects of interruption. We discuss implications of these results for the design of an attention manager. We describe a study on the influence of instant messaging (IM) on ongoing computing tasks. The study both replicates and extends earlier work on the cost of sending notifications at different times and the sensitivity of different tasks to interruption. We investigate alternative hypotheses about the nature of disruption for a list evaluation task, an activity identified as being particularly costly to interrupt. Our findings once again show the generally disruptive effects of IM, especially during fast, stimulus-driven search tasks. In addition, we show that interruptions coming early during a search task are more likely to result in the user forgetting the primary task goal than interruptions that arrive later on. These findings have implications for the design of user interfaces and notification policies that minimize the disruptiveness of notifications.