Human errors in performing security-critical tasks are typically blamed on the complexity of those tasks. However, such errors can also occur because of (possibly unexpected) sensory distractions. A sensory distraction that produces negative effects can be abused by the adversary that controls the environment. Meanwhile, a distraction with positive effects can be artificially introduced to improve user performance. The goal of this work is to explore the effects of visual stimuli on the performance of security-critical tasks. To this end, we experimented with a large number of subjects who were exposed to a range of unexpected visual stimuli while attempting to perform Bluetooth Pairing. Our results clearly demonstrate substantially increased task completion times and markedly lower task success rates. These negative effects are noteworthy, especially, when contrasted with prior results on audio distractions which had positive effects on performance of similar tasks. Experiments were conducted in a novel (fully automated and completely unattended) experimental environment. This yielded more uniform experiments, better scalability and significantly lower financial and logistical burdens. We discuss this experience, including benefits and limitations of the unattended automated experiment paradigm.
O'Malley and Poplawsky @cite_2 argued that sensory noise affects behavioral selectivity. Specifically, while a consistent positive or negative effect on task completion may not occur, a consistent negative effect was observed for tasks that require subjects to react to signals on their periphery. Meanwhile, a consistent positive effect on task completion was observed for tasks that require subjects to react to signals in the center of their field of attention. This leads to the claim that sensory stimulation has the effect of narrowing the subject's area of attention.
@cite_2
52540693
Author(s): Kaczmarek, T; Kobsa, A; Sy, R; Tsudik, G | Abstract: User errors while performing security-critical tasks can lead to undesirable or even disastrous consequences. One major factor influencing mistakes and failures is complexity of such tasks, which has been studied extensively in prior research. Another important issue which hardly received any attention is the impact of both accidental and intended distractions on users performing security-critical tasks. In particular, it is unclear whether, and to what extent, unexpected sensory cues (e.g., auditory or visual) can influence user behavior and or trigger mistakes. Better understanding of the effects of intended distractions will help clarify their role in adversarial models. As part of the research effort described in this paper, we administered a range of naturally occurring -- yet unexpected -- sounds while study participants attempted to perform a security-critical task. We found that, although these auditory cues lowered participants' failure rates, they had no discernible effect on their task completion times. To this end, we overview some relevant literature that explains these somewhat counter-intuitive findings. Conducting a thorough and meaningful study on user errors requires a large number of participants, since errors are typically infrequent and should not be instigated more than once per subject. To reduce the effort of running numerous subjects, we developed a novel experimental setup that was fully automated and unattended. We discuss our experience with this setup and highlight the pros and cons of generalizing its usage. The literature on the effects of noise on monitoring performance shows a disappointing lack of consistency in results. The hypothesis of the present study was that task classification in terms of demands made on the observer should reconcile conflicting findings so that generalizations could be made. Therefore, a study was made of the effects of intermittent or variable noise on vigilance experiments with similar task demands. Twenty-one sensory vigilance studies, selected from 98 visual performance experiments, were analyzed in detail. It appeared that, even when studies possess similar task characteristics, they are hard to compare due to the many types and varieties of the noise variables involved and the measures of performance used. Contradictory results remain. It was concluded that we know nothing about the effects of variable noise on sustained attention, despite the importance of this kind of noise for everyday life. Using this detailed analysis as an illustration, it was suggested that disparate... We present a lexicon-based approach to extracting sentiment from text. The Semantic Orientation CALculator (SO-CAL) uses dictionaries of words annotated with their semantic orientation (polarity and strength), and incorporates intensification and negation. SO-CAL is applied to the polarity classification task, the process of assigning a positive or negative label to a text that captures the text's opinion towards its main subject matter. We show that SO-CAL's performance is consistent across domains and in completely unseen data. Additionally, we describe the process of dictionary creation, and our use of Mechanical Turk to check dictionaries for consistency and reliability. We propose a formal Bayesian definition of surprise to capture subjective aspects of sensory information. Surprise measures how data affects an observer, in terms of differences between posterior and prior beliefs about the world. Only data observations which substantially affect the observer’s beliefs yield surprise, irrespectively of how rare or informative in Shannon’s sense these observations are. We test the framework by quantifying the extent to which humans may orient attention and gaze towards surprising events or items while watching television. To this end, we implement a simple computational model where a low-level, sensory form of surprise is computed by simple simulated early visual neurons. Bayesian surprise is a strong attractor of human attention, with 72 of all gaze shifts directed towards locations more surprising than the average, a figure rising to 84 when focusing the analysis onto regions simultaneously selected by all observers. The proposed theory of surprise is applicable across different spatio-temporal scales, modalities, and levels of abstraction. Task partitioning consists in dividing a task into sub-tasks that can be tackled separately. Partitioning a task might have both positive and negative effects: On the one hand, partitioning might reduce physical interference between workers, enhance exploitation of specialization, and increase efficiency. On the other hand, partitioning may introduce overheads due to coordination requirements. As a result, whether partitioning is advantageous or not has to be evaluated on a case-by-case basis. In this paper we consider the case in which a swarm of robots must decide whether to complete a given task as an unpartitioned task, or utilize task partitioning and tackle it as a sequence of two sub-tasks. We show that the problem of selecting between the two options can be formulated as a multi-armed bandit problem and tackled with algorithms that have been proposed in the reinforcement learning literature. Additionally, we study the implications of using explicit communication between the robots to tackle the studied task partitioning problem. We consider a foraging scenario as a testbed and we perform simulation-based experiments to evaluate the behavior of the system. The results confirm that existing multi-armed bandit algorithms can be employed in the context of task partitioning. The use of communication can result in better performance, but in may also hinder the flexibility of the system. A system and apparatus for visually cuing a human subject wherein an incandescent lamp or lamps (40, 41) are energizable to provide a source of visible illumination. These incandescent lamps have a predetermined rise and fall characteristic. An observation surface (12) is positioned for confronting illumination from the lamps (40, 41) including a pattern through which filtered light of a first intensity may pass, which pattern is configured for visually conveying information to the subject. The pattern is surrounded by a region (54) opaque to illumination, that region extending to a periphery (55). A peripheral surface arrangement (14, 16) is provided extending from the observation surface (12) periphery and is positioned for confronting and transmitting illumination from the lamps (40, 41) at a second intensity selected as greater than the first intensity. Lamps (40, 41) are illuminated to effect the transmission of illumination through the peripheral surface and observation surface in an intermittent fashion at a frequency selected to provide visual stimuli of predetermined temporar pause, p, to evoke a gamma effect with respect to human visual perception of the pattern. Little attention has been paid so far to physiological signals for emotion recognition compared to audiovisual emotion channels such as facial expression or speech. This paper investigates the potential of physiological signals as reliable channels for emotion recognition. All essential stages of an automatic recognition system are discussed, from the recording of a physiological data set to a feature-based multiclass classification. In order to collect a physiological data set from multiple subjects over many weeks, we used a musical induction method that spontaneously leads subjects to real emotional states, without any deliberate laboratory setting. Four-channel biosensors were used to measure electromyogram, electrocardiogram, skin conductivity, and respiration changes. A wide range of physiological features from various analysis domains, including time frequency, entropy, geometric analysis, subband spectra, multiscale entropy, etc., is proposed in order to find the best emotion-relevant features and to correlate them with emotional states. The best features extracted are specified in detail and their effectiveness is proven by classification results. Classification of four musical emotions (positive high arousal, negative high arousal, negative low arousal, and positive low arousal) is performed by using an extended linear discriminant analysis (pLDA). Furthermore, by exploiting a dichotomic property of the 2D emotion model, we develop a novel scheme of emotion-specific multilevel dichotomous classification (EMDC) and compare its performance with direct multiclass classification using the pLDA. An improved recognition accuracy of 95 percent and 70 percent for subject-dependent and subject-independent classification, respectively, is achieved by using the EMDC scheme. We study how the visual catchiness (saliency) of relevant information impacts user engagement metrics such as focused attention and emotion (affect). Participants completed tasks in one of two conditions, where the task-relevant information either appeared salient or non-salient. Our analysis provides insights into relationships between saliency, focused attention, and affect. Participants reported more distraction in the non-salient condition, and non-salient information was slower to find than salient. Lack-of-saliency led to a negative impact on affect, while saliency maintained positive affect, suggesting its helpfulness. Participants reported that it was easier to focus in the salient condition, although there was no significant improvement in the focused attention scale rating. Finally, this study suggests user interest in the topic is a good predictor of focused attention, which in turn is a good predictor of positive affect. These results suggest that enhancing saliency of user-interested topics seems a good strategy for boosting user engagement. Attention can be focused volitionally by “top-down” signals derived from task demands and automatically by “bottom-up” signals from salient stimuli. The frontal and pariet al cortices are involved, but their neural activity has not been directly compared. Therefore, we recorded from them simultaneously in monkeys. Prefrontal neurons reflected the target location first during top-down attention, whereas pariet al neurons signaled it earlier during bottom-up attention. Synchrony between frontal and pariet al areas was stronger in lower frequencies during top-down attention and in higher frequencies during bottom-up attention. This result indicates that top-down and bottom-up signals arise from the frontal and sensory cortex, respectively, and different modes of attention may emphasize synchrony at different frequencies. How do emotions and moods color cognition? In this article, we examine how such reactions influence both judgments and cognitive performance. We argue that many affective influences are due, not to affective reactions themselves, but to the information they carry about value. The specific kind of influence that occurs depends on the focus of the agent at the time. When making evaluative judgments, for example, an agent's positive affect may emerge as a positive attitude toward a person or object. But when an agent focuses on a cognitive task, positive affect may act like feedback about the value of one's approach. As a result, positive affect tends to promote cognitive, relational processes, whereas negative affect tends to inhibit relational processing, resulting in more perceptual, stimulus-specific processing. As a consequence, many textbook phenomena from cognitive psychology occur readily in happy moods, but are inhibited or even absent in sad moods (149). Ambient sensory distractions play a role in security-critical tasks that require human involvement. Adversarial control over human sensory input could broaden the attack surface. To shed light on this issue, a team conducted large-scale experiments that exposed subjects to unexpected audio and visual stimuli while they performed a security-critical task. One of the main challenges in autonomous manipulation is to generate appropriate multi-modal reference trajectories that enable feedback controllers to compute control commands that compensate for unmodeled perturbations and therefore to achieve the task at hand. We propose a data-driven approach to incrementally acquire reference signals from experience and decide online when and to which successive behavior to switch, ensuring successful task execution. We reformulate this online decision making problem as a pair of related classification problems. Both process the current sensor readings, composed from multiple sensor modalities, in real-time (at 30 Hz). Our approach exploits that movement generation can dictate sensor feedback. Thus, enforcing stereotypical behavior will yield stereotypical sensory events which can be accumulated and stored along with the movement plan. Such movement primitives, augmented with sensor experience, are called Associative Skill Memories (ASMs). Sensor experience consists of (real) sensors, including haptic, auditory information and visual information, as well as additional (virtual) features. We show that our approach can be used to teach dexterous tasks, e.g. a bimanual manipulation task on a real platform that requires precise manipulation of relatively small objects. Task execution is robust against perturbation and sensor noise, because our method decides online whether or not to switch to alternative ASMs due to unexpected sensory signals. We introduce a new class of random neural networks in which signals are either negative or positive. A positive signal arriving at a neuron increases its total signal count or potential by one; a negative signal reduces it by one if the potential is positive, and has no effect if it is zero. When its potential is positive, a neuron fires, sending positive or negative signals at random intervals to neurons or to the outside. Positive signals represent excitatory signals and negative signals represent inhibition. We show that this model, with exponential signal emission intervals, Poisson external signal arrivals, and Markovian signal movements between neurons, has a product form leading to simple analytical expressions for the system state. Five normal subjects were tested in a simulated grasping task. A cylindrical container filled with water was placed on the center of a horizontal monitor screen. Subjects used a precision grip formed by the thumb and index finger of their right hand. After a preliminary run during which the container was present, it was replaced by an image of the upper surface of the cylinder appearing on the horizontal computer screen on which the real cylinder was placed during the preliminary run. In each trial the image was marked with two contact points which defined an opposition axis in various orientations with respect to the frontal plane. The subjects' task consisted, once shown a stimulus, of judging as quickly as possible whether the previously experienced action of grasping the container full of water and pouring the water out would be easy, difficult or impossible with the fingers placed according to the opposition axis indicated on the circle. Response times were found to be longer for the grasps judged to be more difficult due to the orientation and position of the opposition axis. In a control experiment, three subjects actually performed the grasps with different orientations and positions of the opposition axis. The effects of these parameters on response time followed the same trends as during simulated movements. This result shows that simulated hand movements take into account the same biomechanical limitations as actually performed movements. We report a user study which compared four virtual environment travel techniques using a divided attention task. Participants used either real walking, gaze-directed, pointing-directed, or torso-directed travel to follow a target through an environment while simultaneously responding to auditory stimuli. In addition to travel technique, we investigated gender as a between-subjects variable and task difficulty (simple or complex) and task type (single or divided) as within-subjects variables. Real walking allowed superior performance over the pointing-directed technique on measures of navigation task performance and recognition of stimuli presented during navigation. This indicates that participants using real walking may have had more spare cognitive capacity to process and encode stimuli than those using pointing-directed travel. We also found a gender-difficulty interaction where males performed worse and responded slower to the attention task when the spatial task was more difficult, but no differences were observed for females between difficulty levels. While these results may be pertinent for the design of virtual environments, the nature and goal of the virtual environment tasks must be carefully considered to determine whether similar effects on performance can be expected under different conditions. Learning rich visual representations often require training on datasets of millions of manually annotated examples. This substantially limits the scalability of learning effective representations as labeled data is expensive or scarce. In this paper, we address the problem of unsupervised visual representation learning from a large, unlabeled collection of images. By representing each image as a node and each nearest-neighbor matching pair as an edge, our key idea is to leverage graph-based analysis to discover positive and negative image pairs (i.e., pairs belonging to the same and different visual categories). Specifically, we propose to use a cycle consistency criterion for mining positive pairs and geodesic distance in the graph for hard negative mining. We show that the mined positive and negative image pairs can provide accurate supervisory signals for learning effective representations using Convolutional Neural Networks (CNNs). We demonstrate the effectiveness of the proposed unsupervised constraint mining method in two settings: (1) unsupervised feature learning and (2) semi-supervised learning. For unsupervised feature learning, we obtain competitive performance with several state-of-the-art approaches on the PASCAL VOC 2007 dataset. For semi-supervised learning, we show boosted performance by incorporating the mined constraints on three image classification datasets. In complex conversation tasks, people react to their interlocutor’s state, such as uncertainty and engagement to improve conversation effectiveness Forbes-Riley and Litman (Adapting to student uncertainty improves tutoring dialogues, pp 33–40, 2009 [2]). If a conversational system reacts to a user’s state, would that lead to a better conversation experience? To test this hypothesis, we designed and implemented a dialog system that tracks and reacts to a user’s state, such as engagement, in real time. We designed and implemented a conversational job interview task based on the proposed framework. The system acts as an interviewer and reacts to user’s disengagement in real-time with positive feedback strategies designed to re-engage the user in the job interview process. Experiments suggest that users speak more while interacting with the engagement-coordinated version of the system as compared to a non-coordinated version. Users also reported the former system as being more engaging and providing a better user experience. Previous studies of eye gaze have shown that when looking at images containing human faces, observers tend to rapidly focus on the facial regions. But is this true of other high-level image features as well? We here investigate the extent to which natural scenes containing faces, text elements, and cell phones - as a suitable control - attract attention by tracking the eye movements of subjects in two types of tasks - free viewing and search. We observed that subjects in free-viewing conditions look at faces and text 16.6 and 11.1 times more than similar regions normalized for size and position of the face and text. In terms of attracting gaze, text is almost as effective as faces. Furthermore, it is difﬁcult to avoid looking at faces and text even when doing so imposes a cost. We also found that subjects took longer in making their initial saccade when they were told to avoid faces text and their saccades landed on a non-face non-text object. We reﬁne a well-known bottom–up computer model of saliency-driven attention that includes conspicuity maps for color, orientation, and intensity by adding high-level semantic information (i.e., the location of faces or text) and demonstrate that this signiﬁcantly improves the ability to predict eye ﬁxations in natural images. Our enhanced model’s predictions yield an area under the ROC curve over 84 for images that contain faces or text when compared against the actual ﬁxation pattern of subjects. This suggests that the primate visual system allocates attention using such an enhanced saliency map. Crowdsourcing relies on the contributions of a large number of workers to accomplish spatial tasks, and it has drawn more attention in recent years. Many crowdsourcing tasks are completed online due to its convenience and efficiency. However, sometimes this traditional method may not work due to special requirements involving actual physical locations. Thus, a new paradigm of data collection, called spatial crowdsourcing, has emerged in the past few years. Spatial crowdsourcing consists of location-specific tasks that require people to physically be at specific locations to complete them. In this article we discuss unique challenges of spatial crowdsourcing, provide a comprehensive view of this new paradigm by introducing the taxonomy, and give future directions. Interest in on-body communication channels is growing as the use of wireless devices increases in medical, consumer and military sensor applications. This paper presents an experimental investigation and analysis of the narrowband on-body propagation channel. This analysis considers each of the factors affecting the channel during a range of stationary and motion activities in different environments with actual wireless mote devices on the body. Use of such motes allows greater freedom in the subject's movements and the inclusion of real-world indoor and outdoor environments in a test sequence. This paper identifies and analyses the effect of the different components of the signal propagation (mean propagation path gain, large-scale fading and small-scale fading) and the cause of the losses and variation due to activities, positions or environmental factors. Our results show the effect on the received signal and the impact of voluntary and involuntary movements, which cause shadowing effects. The analysis also allows us to identify sensor positions on the body that are more reliable and those positions that may require a relay or those that may be suitable for acting as a relay.