proposed a Brain-Computer-Interface (BCI) -based attack in which an adversary is able to infer private information about a user, such as their bank or area-of-living, by analyzing the user's brain activities. However, a key limitation of the above attack is that it is intrusive, requiring user cooperation, and is thus easily detectable and can be reported to other users. In this paper, we identify and analyze a more serious threat for users of BCI devices. We propose a it subliminal attack in which the victim is attacked at the levels below his cognitive perception. Our attack involves exposing the victim to visual stimuli for a duration of 13.3 milliseconds -- a duration usually not sufficient for conscious perception. The attacker analyzes subliminal brain activity in response to these short visual stimuli to infer private information about the user. If carried out carefully, for example by hiding the visual stimuli within screen content that the user expects to see, the attack may remain undetected. As a consequence, the attacker can scale it to many victims and expose them to the attack for a long time. We experimentally demonstrate the feasibility of our subliminal attack via a proof-of-concept study carried out with 27 subjects. We conducted experiments on users wearing Electroencephalography-based BCI devices, and used portrait pictures of people as visual stimuli which were embedded within the background of an innocuous video for a time duration not exceeding 13.3 milliseconds. Our experimental results show that it is feasible for an attacker to learn relevant private information about the user, such as whether the user knows the identity of the person for which the attacker is probing.
Although we have achieved some degree of success in our setup concerning face stimuli, it could be worthwhile to explore other types of stimuli, as the neurological response to faces is in many ways unique from other types of visual processing. Subliminal presentation of faces as opposed to words or randomized dots shows a general greater amplitude of ERP, suggesting that faces are processed differently @cite_20 .
@cite_20
2146234730
Current theories of emotion suggest that threat-related stimuli are first processed via an automatically engaged neural mechanism, which occurs outside conscious awareness. This mechanism operates in conjunction with a slower and more comprehensive process that allows a detailed evaluation of the potentially harmful stimulus (LeDoux, 1998). We drew on the Halgren and Marinkovic (1995) model to examine these processes using event-related potentials (ERPs) within a backward masking paradigm. Stimuli used were faces with fear and neutral (as baseline control) expressions, presented above (supraliminal) and below (subliminal) the threshold for conscious detection. ERP data revealed a double dissociation for the supraliminal versus subliminal perception of fear. In the subliminal condition, responses to the perception of fear stimuli were enhanced relative to neutral for the N2 "excitatory" component, which is thought to represent orienting and automatic aspects of face processing. By contrast, supraliminal perception of fear was associated with relatively enhanced responses for the late P3 "inhibitory" component, implicated in the integration of emotional processes. These findings provide evidence in support of Halgren and Marinkovic's temporal model of emotion processing, and indicate that the neural mechanisms for appraising signals of threat may be initiated, not only automatically, but also without the need for conscious detection of these signals. This experiment provides brain event-related potential (ERP) evidence for differential processing of visually presented pleasant and unpleasant affectively valent words (mood adjectives) for both supraliminal (40 ms) and subliminal (unmasked, 1 ms) stimulus durations. Unpleasant words elicited a more positive amplitude than pleasant words in both durations. ERP components (P1, N1, P2, P3, and a late positive potential; LP) were measured at six electrode sites (F3, F4, P3, P4, CzPz, Oz). ERPs to subliminal stimuli demonstrated differences between pleasant and unpleasant words in the left hemisphere across all measured components. Supraliminal processing showed similar differences in the left hemisphere for early components (P1 and N1), but bilateral differences for late components (P3 and LP). Activity in the P2 time window was associated with the divergence between supraliminal and subliminal affective responses. Implications for the study of affect and consciousness are discussed. Hair is a feature of the head that frequently changes in different situations. For this reason much research in the area of face perception has employed stimuli without hair. To investigate the effect of the presence of hair we used faces with and without hair in a recognition task. Participants took part in trials in which the state of the hair either remained consistent (Same) or switched between learning and test (Switch). It was found that in the Same trials performance did not differ for stimuli presented with and without hair. This implies that there is sufficient information in the internal features of the face for optimal performance in this task. It was also found that performance in the Switch trials was substantially lower than in the Same trials. This drop in accuracy when the stimuli were switched suggests that faces are represented in a holistic manner and that manipulation of the hair causes disruption to this, with implications for the interpretation of some previous studies. ABSTRACT How is visual information linked to aesthetic experience, and what factors determine whether an individual finds a particular visual experience pleasing? We have previously shown that individuals aesthetic responses are not determined by objective image features but are instead a function of internal, subjective factors that are shaped by a viewers personal experience. Ye t for many classes of stimuli, culturally shared semantic associations give rise to similar aesthetic taste across people. In this paper, we investigated factors that govern whether a set of observers will agree in which images are preferred, or will instead exhibit more per sonalized aesthetic preferences. In a series of experiments, observers were asked to make aesthetic judgments for different categories of visual stimuli that are commonly evaluated in an aesthetic manner (faces, natural landscapes, architecture or artwork). By measuring agreement across observers, this method was able to reveal instances of highly individualistic preferences. We found that observers showed high agreement on their preferences for images of faces and landscapes, but much lower agreement for images of artwork and architecture. In addition, we found higher agreement for he terosexual males making judgments of beautiful female faces than of beautiful male faces. These results suggest that preferences for stimulus categories that carry evolutionary significance (landscapes and faces) come to rely on similar information across individuals, whereas preferences for artifacts of human culture such as architecture and artwork, which have fewer basic-level category distinctions and reduced behavioral relevance, rely on a more personalized set of attributes. Keywords: Aesthetics, individual differences, visual preference, artwork, landscapes, faces, architecture Abstract In 2 experiments, the development of auditory selective attention in children was assessed. The participants, aged 5, 7, and 9 years, responded to target stimuli in the left ear for 1 series (ignoring the standard stimuli) and in the right ear for the other series. In Experiment 1, event-related potentials (ERPs) evoked by the auditory stimuli were recorded from frontal, central, and pariet al sites. The 9-year-olds showed a greater processing negativity (Nd) to the attended channel compared with the 5-year-olds. Both 7-and 9-year-olds showed significantly larger amplitudes for the P3 component of ERPs in the attended vs. ignored condition. Behaviorally, the 5-year-olds made fewer hits and more false alarms than did older children, and the 9-year-olds made significantly more false alarms than did the 7-year-olds. The results of Experiment 2 showed that the detriment in the performance of the 9-year-olds was a result of task parameters. The inability of 5-year-olds to attend selectively appears to ... A common assumption in studies of interruptions is that one is focused in an activity and then distracted by other stimuli. We take the reverse perspective and examine whether one might first be in an attentional state that makes one susceptible to communications typically associated with distraction. We explore the confluence of multitasking and workplace communications from three temporal perspectives -- prior to an interaction, when tasks and communications are interleaved, and at the end of the day. Using logging techniques and experience sampling, we observed 32 employees in situ for five days. We found that certain attentional states lead people to be more susceptible to particular types of interaction. Rote work is followed by more Facebook or face-to-face interaction. Focused and aroused states are followed by more email. The more time in email and face-fo-face interaction, and the more total screen switches, the less productive people feel at the day's end. We present the notion of emotional homeostasis along with new directions for multitasking research. This study examined the effect of attention engagement to compound auditory-visual stimuli on the modification of the startle blink reflex in infants. Infants at 8, 14, 20, or 26 weeks of age were presented with interesting audiovisual stimuli. After stimulus onset, at delays defined by heart rate changes known to be associated with sustained attention or attention disengagement, blink reflexes were elicited by visual or auditory stimuli. Blink amplitude to either visual or auditory stimuli was enhanced when the infants were engaged in attention to the foreground auditory-visual stimuli relative to control trials with no foreground patterns. This enhancement of the blink amplitude increased from 8 to 26 weeks of age. In contrast to selective modality enhancement for single-modality foreground stimuli, these results show that these multimodal stimuli engage both visual and auditory attention systems in this age range. Descriptors: Infants, Attention, Heart rate, Blink reflex, Multimodal stimuli Manipulated versions of three-dimensional faces that have different profiles, but almost the same appearance in frontal views, provide a novel way to investigate if and how humans use class-specific knowledge to infer depth from images of faces. After seeing a frontal view, participants have to select the profile that matches that view. The profiles are original (ground truth), average, random other, and two solutions computed with a linear face model (3D Morphable Model). One solution is based on 2D vertex positions, the other on pixel colors in the frontal view. The human responses demonstrate that humans neither guess nor just choose the average profile. The results also indicate that humans actually use the information from the front view, and not just rely on the plausibility of the profiles per se. All our findings are perfectly consistent with a correlation-based inference in a linear face model. The results also verify that the 3D reconstructions from our computational algorithms (stimuli 4 and 5) are similar to what humans expect, because they are chosen to be the true profile equally often as the ground-truth profiles. Our experiments shed new light on the mechanisms of human face perception and present a new quality measure for 3D reconstruction algorithms. By adding the same component (e.g., glasses) to two stimuli (e.g., schematic faces) or to one stimulus only, it is possible to assess the impact of that component as a common or as a distinctive feature. A formal procedure, based on the contrast model (A. Tversky, 1977, Psychological Review, 84, 327–352), for estimating the relative weight of common to distinctive features from similarity judgments between separable stimuli with independent components, was developed. The results show that in verbal stimuli (e.g., descriptions of persons, meals, trips) common features loom larger than distinctive features. On the other hand, in pictorial stimuli (e.g., schematic faces, landscapes) distinctive features loom larger than common features. Verbal descriptions of pictorial stimuli were evaluated like other verbal stimuli and unlike their pictorial counterparts. In conceptual comparisons, the relative weight of common to distinctive features was higher in judgments of similarity than in judgments of dissimilarity. White noise techniques have been used widely to investigate sensory systems in both vertebrates and invertebrates. White noise stimuli are powerful in their ability to rapidly generate data that help the experimenter decipher the spatio-temporal dynamics of neural and behavioral responses. One type of white noise stimuli, maximal length shift register sequences (m-sequences), have recently become particularly popular for extracting response kernels in insect motion vision. We here use such m-sequences to extract the impulse responses to figure motion in hoverfly lobula plate tangential cells (LPTCs). Figure motion is behaviorally important and many visually guided animals orient towards salient features in the surround. We show that LPTCs respond robustly to figure motion in the receptive field. The impulse response is scaled down in amplitude when the figure size is reduced, but its time course remains unaltered. However, a low contrast stimulus generates a slower response with a significantly longer time-to-peak and half-width. Impulse responses in females have a slower time-to-peak than males, but are otherwise similar. Finally we show that the shapes of the impulse response to a figure and a widefield stimulus are very similar, suggesting that the figure response could be coded by the same input as the widefield response. We describe a probabilistic method for identifying characters in TV series or movies. We aim at labeling every character appearance, and not only those where a face can be detected. Consequently, our basic unit of appearance is a person track (as opposed to a face track). We model each TV series episode as a Markov Random Field, integrating face recognition, clothing appearance, speaker recognition and contextual constraints in a probabilistic manner. The identification task is then formulated as an energy minimization problem. In order to identify tracks without faces, we learn clothing models by adapting available face recognition results. Within a scene, as indicated by prior analysis of the temporal structure of the TV series, clothing features are combined by agglomerative clustering. We evaluate our approach on the first 6 episodes of The Big Bang Theory and achieve an absolute improvement of 20 for person identification and 12 for face recognition. Recognition of unfamiliar faces is susceptible to image differences caused by angular sizes subtended from the face to the camera. Research on perception of cubes suggests that apparent distortions of a shape due to large camera angle are correctable by placing the observer at the centre of projection, especially when visibility of the picture surface is low (Yang and Kubovy, 1999 Perception & Psychophysics 61 456^467). To explore the implication of this finding for face perception, observers performed recognition and matching tasks where face images with reduced visibility of picture surface were shown with observers either at the centre of projec- tion or at other viewpoints. The results show that, unlike perception of cubes, the effect of perspective transformation on face recognition is largely unaffected by the centre of projection. Furthermore, the use of perspective cues is not affected by textured surfaces. The limitation of perspective in restoring 3-D information of faces suggests a stronger role for image-based, rather than model-based, processes in recognition of unfamiliar faces. In this paper, we present a neural network-based face detection system. Unlike similar systems which are limited to detecting upright, frontal faces, this system detects faces at any degree of rotation in the image plane. The system employs multiple networks; a "router" network first processes each input window to determine its orientation and then uses this information to prepare the window for one or more "detector" networks. We present the training methods for both types of networks. We also perform sensitivity analysis on the networks, and present empirical results on a large test set. Finally, we present preliminary results for detecting faces rotated out of the image plane, such as profiles and semi-profiles. MAHNOB-HCI is a multimodal database recorded in response to affective stimuli with the goal of emotion recognition and implicit tagging research. A multimodal setup was arranged for synchronized recording of face videos, audio signals, eye gaze data, and peripheral central nervous system physiological signals. Twenty-seven participants from both genders and different cultural backgrounds participated in two experiments. In the first experiment, they watched 20 emotional videos and self-reported their felt emotions using arousal, valence, dominance, and predictability as well as emotional keywords. In the second experiment, short videos and images were shown once without any tag and then with correct or incorrect tags. Agreement or disagreement with the displayed tags was assessed by the participants. The recorded videos and bodily responses were segmented and stored in a database. The database is made available to the academic community via a web-based system. The collected data were analyzed and single modality and modality fusion results for both emotion recognition and implicit tagging experiments are reported. These results show the potential uses of the recorded modalities and the significance of the emotion elicitation protocol. The task of the emotion recognition in the wild (EmotiW) Challenge is to assign one of seven emotions to short video clips extracted from Hollywood style movies. The videos depict acted-out emotions under realistic conditions with a large degree of variation in attributes such as pose and illumination, making it worthwhile to explore approaches which consider combinations of features from multiple modalities for label assignment. In this paper we present our approach to learning several specialist models using deep learning techniques, each focusing on one modality. Among these are a convolutional neural network, focusing on capturing visual information in detected faces, a deep belief net focusing on the representation of the audio stream, a K-Means based "bag-of-mouths" model, which extracts visual features around the mouth region and a relational autoencoder, which addresses spatio-temporal aspects of videos. We explore multiple methods for the combination of cues from these modalities into one common classifier. This achieves a considerably greater accuracy than predictions from our strongest single-modality classifier. Our method was the winning submission in the 2013 EmotiW challenge and achieved a test set accuracy of 47.67 on the 2014 dataset. This study investigated visual cues to age by using facial composites which blend shape and colour information from multiple faces. Baseline measurements showed that perceived age of adult male faces is on average an accurate index of their chronological age over the age range 20-60 years. Composite images were made from multiple images of different faces by averaging face shape and then blending red, green and blue intensity (RGB colour) across comparable pixels. The perceived age of these composite or blended images depended on the age bracket of the component faces. Blended faces were, however, rated younger than their component faces, a trend that became more marked with increased component age. The techniques used provide an empirical definition of facial changes with age that are biologically consistent across a sample population. The perceived age of a blend of old faces was increased by exaggerating the RGB colour differences of each pixel relative to a blend of young faces. This effect on perceived age was not attributable to enhanced contrast or colour saturation. Age-related visual cues defined from the differences between blends of young and old faces were applied to individual faces. These transformations increased perceived age. An analysis is conducted on the robustness of measures of centrality in the face of random error in the network data. We use random networks of varying sizes and densities and subject them (separately) to four kinds of random error in varying amounts. The types of error are edge deletion, node deletion, edge addition, and node addition. The results show that the accuracy of centrality measures declines smoothly and predictably with the amount of error. This suggests that, for random networks and random error, we shall be able to construct confidence intervals around centrality scores. In addition, centrality measures were highly similar in their response to error. Dense networks were the most robust in the face of all kinds of error except edge deletion. For edge deletion, sparse networks were more accurately measured. Face images captured in different spectral bands, e.g. , in visual (VIS) and near infrared (NIR), are said to be heterogeneous. Although a person's face looks different in heterogeneous images, it should be classified as being from the same individual. In this paper, we present a new method, called face analogy , in the analysis-by-synthesis framework, for heterogeneous face mapping, that is, transforming face images from one type to another, and thereby performing heterogeneous face matching. Experiments show promising results. Preface 1. Introduction 2. Some context for Context Analysis: a view of the origins of structural studies of face-to-face interaction 3. Some functions of gaze direction in two-person conversation 4. Movement co-ordination in social interaction 5. Some functions of the face in a kissing round 6. A description of some human greetings 7. Spatial organisation in social encounters: the F-formation system 8. Behavioural foundations for the process of frame-attunement in face-to-face interaction List of films cited References Index. The varying quality of face images is an important challenge that limits the eﬀectiveness of face recognition technology when applied in real-world applications. Existing face image databases do not consider the eﬀect of distortions that commonly occur in real-world environments. This database (QLFW) represents an initial attempt to provide a set of labeled face images spanning the wide range of quality, from no perceived impairment to strong perceived impairment for face detection and face recognition applications. Types of impairment include JPEG2000 compression, JPEG compression, additive white noise, Gaussian blur and contrast change. Subjective experiments are conducted to assess the perceived visual quality of faces under diﬀerent levels and types of distortions and also to assess the human recognition performance under the considered distortions. One goal of this work is to enable automated performance evaluation of face recognition technologies in the presence of diﬀerent types and levels of visual distortions. This will consequently enable the development of face recognition systems that can operate reliably on real-world visual content in the presence of real-world visual distortions. Another goal is to enable the development and assessment of visual quality metrics for face images and for face detection and recognition applications.