Human errors in performing security-critical tasks are typically blamed on the complexity of those tasks. However, such errors can also occur because of (possibly unexpected) sensory distractions. A sensory distraction that produces negative effects can be abused by the adversary that controls the environment. Meanwhile, a distraction with positive effects can be artificially introduced to improve user performance. The goal of this work is to explore the effects of visual stimuli on the performance of security-critical tasks. To this end, we experimented with a large number of subjects who were exposed to a range of unexpected visual stimuli while attempting to perform Bluetooth Pairing. Our results clearly demonstrate substantially increased task completion times and markedly lower task success rates. These negative effects are noteworthy, especially, when contrasted with prior results on audio distractions which had positive effects on performance of similar tasks. Experiments were conducted in a novel (fully automated and completely unattended) experimental environment. This yielded more uniform experiments, better scalability and significantly lower financial and logistical burdens. We discuss this experience, including benefits and limitations of the unattended automated experiment paradigm.
@cite_1 discovered that subject's performance in secure device pairing could be improved if it were to be scored. In other words, notifying subjects about their performance score resulted in fewer errors.
@cite_1
15257636
We explore the use of extrinsic motivation to improve the state of user-centered security mechanisms. Specifically, we study applications of scores as user incentives in the context of secure device pairing. We develop a scoring functionality that can be integrated with traditional pairing approaches. We then report on a usability study that we performed to evaluate the effect of scoring on the performance of users in comparison operations. Our results demonstrate that individuals are likely to commit fewer errors and show more acceptance when working with the scoring based pairing approach. Framing pairing as a game and providing feedback to users in the form of a score is an efficient way to improve pairing security, particularly among users such as children who may not be aware of the consequences of their decisions while performing security tasks. Face recognition technologies have seen dramatic improvements in performance over the past decade, and such systems are now widely used for security and commercial applications. Since recognizing faces is a task that humans are understood to be very good at, it is common to want to compare automatic face recognition (AFR) and human face recognition (HFR) in terms of biometric performance. This paper addresses this question by: 1) conducting verification tests on volunteers (HFR) and commercial AFR systems and 2) developing statistical methods to support comparison of the performance of different biometric systems. HFR was tested by presenting face-image pairs and asking subjects to classify them on a scale of ldquoSame,rdquo ldquoProbably Same,rdquo ldquoNot sure,rdquo ldquoProbably Different,rdquo and ldquoDifferentrdquo; the same image pairs were presented to AFR systems, and the biometric match score was measured. To evaluate these results, two new statistical evaluation techniques are developed. The first is a new way to normalize match-score distributions, where a normalized match score is calculated as a function of the angle from a representation of [false match rate, false nonmatch rate] values in polar coordinates from some center. Using this normalization, we develop a second methodology to calculate an average detection error tradeoff (DET) curve and show that this method is equivalent to direct averaging of DET data along each angle from the center. This procedure is then applied to compare the performance of the best AFR algorithms available to us in the years 1999, 2001, 2003, 2005, and 2006, in comparison to human scores. Results show that algorithms have dramatically improved in performance over that time. In comparison to the performance of the best AFR system of 2006, 29.2 of human subjects performed better, while 37.5 performed worse. How accurately are people able to use the absolute category rating (ACR) 5-level scale? Put another way, how repeatable are an individual subject’s scores? Several subjective experiments have asked subjects to rate the same sequences a couple of times. Analyses indicate that none of the subjects exactly repeated their prior scores for these sequences. We would like to better understand this imperfection. This paper uses ACR subjective video quality tests to explore the precision of subjective ratings. To make formal measurements possible, we propose a theoretical subject model that is the main contribution of this paper. The proposed subject model indicates three major factors that influence accuracy: subject bias, subject inaccuracy, and stimulus scoring difficulty. These appear to be separate random effects and their existence is a reason why none of the subjects were able to perfectly repeat scores. There are three key consequences. First, subject scoring behavior includes a random component that spans approximately half of the rating scale. Second, the sensitivity and accuracy of most subjective analyses can be improved if the subject scores are normalized by removing subject bias. Third, to some extent, multiple subjects can be replaced with a single subject who rates each sequence multiple times. The question of aggregating pair-wise comparisons to obtain a global ranking over a collection of objects has been of interest for a very long time: be it ranking of online gamers (e.g. MSR's TrueSkill system) and chess players, aggregating social opinions, or deciding which product to sell based on transactions. In most settings, in addition to obtaining a ranking, finding scores' for each object (e.g. player's rating) is of interest for understanding the intensity of the preferences. In this paper, we propose Rank Centrality, an iterative rank aggregation algorithm for discovering scores for objects (or items) from pair-wise comparisons. The algorithm has a natural random walk interpretation over the graph of objects with an edge present between a pair of objects if they are compared; the score, which we call Rank Centrality, of an object turns out to be its stationary probability under this random walk. To study the efficacy of the algorithm, we consider the popular Bradley-Terry-Luce (BTL) model (equivalent to the Multinomial Logit (MNL) for pair-wise comparisons) in which each object has an associated score which determines the probabilistic outcomes of pair-wise comparisons between objects. In terms of the pair-wise marginal probabilities, which is the main subject of this paper, the MNL model and the BTL model are identical. We bound the finite sample error rates between the scores assumed by the BTL model and those estimated by our algorithm. In particular, the number of samples required to learn the score well with high probability depends on the structure of the comparison graph. When the Laplacian of the comparison graph has a strictly positive spectral gap, e.g. each item is compared to a subset of randomly chosen items, this leads to dependence on the number of samples that is nearly order-optimal. The generation of word hypotheses for segmentation-free word spotting on document level is usually subject to heuristic expert design. This involves strong assumptions about the visual appearance of text in the document images. In this paper we propose to generate hypotheses with text detectors. In order to do so, we present three detectors that are based on SIFT contrast scores, CNN region classification scores and attribute activation maps. The uncertainty in the detector scores is modeled with the extremal regions method. Retrieving word hypotheses is based on PHOC representations which we compute with the TPP-PHOCNet. We evaluate our method on the George Washington dataset and the ICFHR 2016 KWS competition benchmarks. In the evaluation we show that high word detection rates can be achieved. This is a prerequisite for high retrieval performance that is competitive with the state-of-the-art. Secure Device Pairing is the bootstrapping of secure communication between two previously unassociated devices over a wireless channel. The human-imperceptible nature of wireless communication, lack of any prior security context, and absence of a common trust infrastructure open the door for Man-in-the-Middle (aka Evil Twin) attacks. A number of methods have been proposed to mitigate these attacks, each requiring user assistance in authenticating information exchanged over the wireless channel via some human-perceptible auxiliary channels, e.g., visual, acoustic or tactile. In this paper, we present results of the first comprehensive and comparative study of eleven notable secure device pairing methods. Usability measures include: task performance times, ratings on System Usability Scale (SUS), task completion rates, and perceived security. Study subjects were controlled for age, gender and prior experience with device pairing. We present overall results and identify problematic methods for certain classes of users as well as methods best-suited for various device configurations. In recent years, the performance of face verification systems has significantly improved using deep convolutional neural networks (DCNNs). A typical pipeline for face verification includes training a deep network for subject classification with softmax loss, using the penultimate layer output as the feature descriptor, and generating a cosine similarity score given a pair of face images. The softmax loss function does not optimize the features to have higher similarity score for positive pairs and lower similarity score for negative pairs, which leads to a performance gap. In this paper, we add an L2-constraint to the feature descriptors which restricts them to lie on a hypersphere of a fixed radius. This module can be easily implemented using existing deep learning frameworks. We show that integrating this simple step in the training pipeline significantly boosts the performance of face verification. Specifically, we achieve state-of-the-art results on the challenging IJB-A dataset, achieving True Accept Rate of 0.909 at False Accept Rate 0.0001 on the face verification protocol. Additionally, we achieve state-of-the-art performance on LFW dataset with an accuracy of 99.78 , and competing performance on YTF dataset with accuracy of 96.08 . We propose a novel algorithm for inducing semantic taxonomies. Previous algorithms for taxonomy induction have typically focused on independent classifiers for discovering new single relationships based on hand-constructed or automatically discovered textual patterns. By contrast, our algorithm flexibly incorporates evidence from multiple classifiers over heterogenous relationships to optimize the entire structure of the taxonomy, using knowledge of a word's coordinate terms to help in determining its hypernyms, and vice versa. We apply our algorithm on the problem of sense-disambiguated noun hyponym acquisition, where we combine the predictions of hypernym and coordinate term classifiers with the knowledge in a preexisting semantic taxonomy (WordNet 2.1). We add 10,000 novel synsets to WordNet 2.1 at 84 precision, a relative error reduction of 70 over a non-joint algorithm using the same component classifiers. Finally, we show that a taxonomy built using our algorithm shows a 23 relative F-score improvement over WordNet 2.1 on an independent testset of hypernym pairs. The ability to control a prosthetic device directly from the neocortex has been demonstrated in rats, monkeys and humans. Here we investigate whether neural control can be accomplished in situations where (1) subjects have not received prior motor training to control the device (na¨ user) and (2) the neural encoding of movement parameters in the cortex is unknown to the prosthetic device (nacontroller). By adopting a decoding strategy that identifies and focuses on units whose firing rate properties are best suited for control, we show that na¨ subjects mutually adapt to learn control of a neural prosthetic system. Six untrained Long-Evans rats, implanted with silicon micro-electrodes in the motor cortex, learned cortical control of an auditory device without prior motor characterization of the recorded neural ensemble. Single- and multi-unit activities were decoded using a Kalman filter to represent an audio 'cursor' (90 ms tone pips ranging from 250 Hz to 16 kHz) which subjects controlled to match a given target frequency. After each trial, a novel adaptive algorithm trained the decoding filter based on correlations of the firing patterns with expected cursor movement. Each behavioral session consisted of 100 trials and began with randomized decoding weights. Within 7 ± 1.4 (mean ± SD) sessions, all subjects were able to significantly score above chance ( P< 0.05, randomization method) in a fixed target paradigm. Training lasted 24 sessions in which both the behavioral performance and signal to noise ratio of the peri-event histograms increased significantly ( P< 0.01, ANOVA). Two rats continued training on a more complex task using a bilateral, two-target control paradigm. Both subjects were able to significantly discriminate the target tones ( P< 0.05, Z-test), while one subject demonstrated control above chance ( P< 0.05, Z-test) after 12 sessions and continued improvement with many sessions achieving over 90 correct targets. Dynamic analysis of binary trial responses indicated that early learning for this subject occurred during session 6. This study demonstrates that subjects can learn to generate neural control signals that are well suited for use with external devices without prior experience or training. Human figures have been animated using a variety of geometric models, including stick figures, polygonal models and NURBS-based models with muscles, flexible skin or clothing. This paper reports on experimental results indicating that a viewer's perception of motion characteristics is affected by the geometric model used for rendering. Subjects were shown a series of paired motion sequences and asked if the two motions in each pair were the same or different. The motion sequences in each pair were rendered using the same geometric model. For the three types of motion variation tested, sensitivity scores indicate that subjects were better able to observe changes with the polygonal model than they were with the stick-figure model. We performed a laboratory experiment to study the privacy tradeoff offered by Facebook Connect: disclosing Facebook profile data to third-party websites for the convenience of logging in without creating separate accounts. We controlled for trustworthiness and amount of information each website requested, as well as the consent dialog layout. We discovered that these factors had no observable effects, likely because participants did not read the dialogs. Yet, 15 still refused to use Facebook Connect, citing privacy concerns. A likely explanation for subjects ignoring the dialogs while also understanding the privacy tradeoff - our exit survey indicated that 88 broadly understood what data would be collected - is that subjects were already familiar with the dialogs prior to the experiment. We discuss how our results demonstrate informed consent, but also how habituation prevented subjects from understanding the nuances between individual websites' data collection policies. The Box and Block Test, a test of manual dexterity, has been used by occupational therapists and others to evaluate physically handicapped individuals. Because the test lacked normative data for adults, the results of the test have been interpreted subjectively. The purpose of this study was to develop normative data for adults. Test subjects were 628 Normal adults (310 males and 318 females) from the seven-county Milwaukee area. Data on males and females 20 to 94 years old were divided into 12 age groups. Means, standard deviations, standard error, and low and high scores are reported for each five-year age group. These data will enable clinicians to objectively compare a patient's score to a normal population parameter. Subjective judgments, an essential information source for science and policy, are problematic because there are no public criteria for assessing judgmental truthfulness. I present a scoring method for eliciting truthful subjective data in situations where objective truth is unknowable. The method assigns high scores not to the most common answers but to the answers that are more common than collectively predicted, with predictions drawn from the same population. This simple adjustment in the scoring criterion removes all bias in favor of consensus: Truthful answers maximize expected score even for respondents who believe that their answer represents a minority view. We propose a new behavioral biometric modality based on multi-touch gestures. We define a canonical set of multi-touch gestures based on the movement characteristics of the palm and fingertips being used to perform the gesture. We developed an algorithm to generate and verify multi-touch gesture templates. We tested our techniques on a set of 22 different gestures. Employing a matching algorithm for a multi-touch verification system with a k-NN classifier we achieved 1.28 Equal Error Rate (EER). With score-based classifiers where only the first five samples of a genuine subject were considered as templates, we achieved 4.46 EER. Further, with the combination of three commonly used gestures: pinch, zoom, and rotate, using all five fingers, 1.58 EER was achieved using a score-based classifier. These results are encouraging and point to the possibility of touch based biometric systems in real world applications like user verification and active authentication. In this paper, we investigate the use of recurrent neural networks (RNNs) in the context of search-based online advertising. We use RNNs to map both queries and ads to real valued vectors, with which the relevance of a given (query, ad) pair can be easily computed. On top of the RNN, we propose a novel attention network, which learns to assign attention scores to different word locations according to their intent importance (hence the name DeepIntent). The vector output of a sequence is thus computed by a weighted sum of the hidden states of the RNN at each word according their attention scores. We perform end-to-end training of both the RNN and attention network under the guidance of user click logs, which are sampled from a commercial search engine. We show that in most cases the attention network improves the quality of learned vector representations, evaluated by AUC on a manually labeled dataset. Moreover, we highlight the effectiveness of the learned attention scores from two aspects: query rewriting and a modified BM25 metric. We show that using the learned attention scores, one is able to produce sub-queries that are of better qualities than those of the state-of-the-art methods. Also, by modifying the term frequency with the attention scores in a standard BM25 formula, one is able to improve its performance evaluated by AUC. Simple quality metrics such as PSNR are known to not correlate well with subjective quality when tested across a wide spectrum of video content or quality regime. Recently, efforts have been made in designing objective quality metrics trained on subjective data, demonstrating better correlation with video quality perceived by human. Clearly, the accuracy of such a metric heavily depends on the quality of the subjective data that it is trained on. In this paper, we propose a new approach to recover subjective quality scores from noisy raw measurements, by jointly estimating the subjective quality of impaired videos, the bias and consistency of test subjects, and the ambiguity of video contents all together. Compared to previous methods which partially exploit the subjective information, our approach is able to exploit the information in full, yielding better handling of outliers without the need for z-scoring or subject rejection. It also handles missing data more gracefully. Lastly, as side information, it provides interesting insights on the test subjects and video contents. This paper describes an experiment to assess the influence of immersion on performance in immersive virtual environments. The task involved Tri-Dimensional Chess, and required subjects to reproduce on a real chess board the state of board learned from a sequence of moves witnessed in a virtual environment. Twenty four subjects were allocated to a factorial design consisting of two levels of immersion (exocentric screen based, and egocentric HMD based), and two kinds of environment (plain and realistic). The results suggest that egocentric subjects performed better than exocentric, and those in the more realistic environment performed better than those in the less realistic environment. Previous knowledge of chess, and amount of virtual practice were also significant, and may be considered as control variables to equalise these factors amongst the subjects. Other things being equal, males remembered the moves better than females, although female performance improved with higher spatial ability test score. The paper also attempts to clarify the relationship between immersion, presence and performance, and locates the experiment within such a theoretical framework. We present two systems created for SemEval2016s Task 11: Complex Word Identification. Our two systems, a regression tree and decision tree, were trained with a word’s unigram and lemma word counts, average ageof-acquisition, and a measure of concreteness. The systems ranked 5th and 6th, respectively, on the test set by G-score (the harmonic mean between accuracy and recall). With the regression tree’s predictions earning a G-score of 0.766, and the decision tree’s earning 0.765, the two systems scored within 1 percent of the score of the best-performing system in the task. We measure media bias by estimating ideological scores for several major media outlets. To compute this, we count the times that a particular media outlet cites various think tanks and policy groups, and then compare this with the times that members of Congress cite the same groups. Our results show a strong liberal bias: all of the news outlets we examine, except Fox News' Special Report and the Washington Times, received scores to the left of the average member of Congress. Consistent with claims made by conservative critics, CBS Evening News and the New York Times received scores far to the left of center. The most centrist media outlets were PBS NewsHour, CNN's Newsnight, and ABC's Good Morning America; among print outlets, USA Today was closest to the center. All of our findings refer strictly to news content; that is, we exclude editorials, letters, and the like. "The editors in Los Angeles killed the story. They told Witcover that it didn't ‘come off’ and that it was an ‘opinion’ story.… The solution was simple, they told him. All he had to do was get other people to make the same points and draw the same conclusions and then write the article in their words" (emphasis in original). Timothy Crouse, Boys on the Bus [1973, p. 116]. In this paper, we address referring expression comprehension: localizing an image region described by a natural language expression. While most recent work treats expressions as a single unit, we propose to decompose them into three modular components related to subject appearance, location, and relationship to other objects. This allows us to flexibly adapt to expressions containing different types of information in an end-to-end framework. In our model, which we call the Modular Attention Network (MAttNet), two types of attention are utilized: language-based attention that learns the module weights as well as the word phrase attention that each module should focus on; and visual attention that allows the subject and relationship modules to focus on relevant image components. Module weights combine scores from all three modules dynamically to output an overall score. Experiments show that MAttNet outperforms previous state-of-art methods by a large margin on both bounding-box-level and pixel-level comprehension tasks.