Keystroke dynamics have been extensively used in psycholinguistic and writing research to gain insights into cognitive processing. But do keystroke logs contain actual signal that can be used to learn better natural language processing models? We postulate that keystroke dynamics contain information about syntactic structure that can inform shallow syntactic parsing. To test this hypothesis, we explore labels derived from keystroke logs as auxiliary task in a multi-task bidirectional Long Short-Term Memory (bi-LSTM). Our results show promising results on two shallow syntactic parsing tasks, chunking and CCG supertagging. Our model is simple, has the advantage that data can come from distinct sources, and produces models that are significantly better than models trained on the text annotations alone.
A recent related line of work explores eye tracking data to inform sentence compression @cite_18 and induce part-of-speech @cite_0 . Similarly, there are recent studies that predict fMRI activation from reading @cite_14 or use fMRI data for POS induction @cite_7 . The distinct advantage of keystroke dynamics is that it is easy to get, non-expensive and non-intrusive.
@cite_0 @cite_18 @cite_14 @cite_7
2043464897 1966461713 2013150886 2168820925
Abstract In this study, we investigated the spatio-temporal patterns of category-specific cortical activation elicited by the visual presentation of words whose meaning relates to a color or to a visual form or shape. We recorded the event-related potentials (ERPs) of ten healthy, right-handed volunteers while they passively read words presented tachistoscopically. As early as 150 ms after stimulus onset, the ERPs revealed significant neurophysiological differences between words and strings of hash marks. Around 200 ms after stimulus onset, we found significant differences in the ERPs elicited by color- and form-related words. We used minimum norm current estimates to investigate the spatial location of these differences. This revealed that, at the 150 ms peak, the activation advantage of words over sequences of hash marks was located in a left posterior area, proximal to what has previously been called the visual word form area. At the 200 ms peak, the advantage of words related to colors over words related to forms seemed to reside in temporal cortical areas, whereas the form-related words elicited greater activation in frontal areas than color-related words. These results provide evidence for early access to detailed category-specific representations of word meaning, with subtle differences in meaning being reflected in the activation of different cortical areas, as early as 200 ms after stimulus presentation. In line with previous studies, these differences can be related to the areas involved in the conceptual processing of sensory (visual) and action-related information. Generally, various visual media are unequally memorable by the human brain. This paper looks into a new direction of modeling the memorability of video clips and automatically predicting how memorable they are by learning from brain functional magnetic resonance imaging (fMRI). We propose a novel computational framework by integrating the power of low-level audiovisual features and brain activity decoding via fMRI. Initially, a user study experiment is performed to create a ground truth database for measuring video memorability and a set of effective low-level audiovisual features is examined in this database. Then, human subjects’ brain fMRI data are obtained when they are watching the video clips. The fMRI-derived features that convey the brain activity of memorizing videos are extracted using a universal brain reference system. Finally, due to the fact that fMRI scanning is expensive and time-consuming, a computational model is learned on our benchmark dataset with the objective of maximizing the correlation between the low-level audiovisual features and the fMRI-derived features using joint subspace learning. The learned model can then automatically predict the memorability of videos without fMRI scans. Evaluations on publically available image and video databases demonstrate the effectiveness of the proposed framework. Abstract Functional magnetic resonance imaging (fMRI) can provide maps of brain activation with millimeter spatial resolution but is limited in its temporal resolution to the order of seconds. Here, we describe a technique that combines structural and functional MRI with magnetoencephalography (MEG) to obtain spatiotemporal maps of human brain activity with millisecond temporal resolution. This new technique was used to obtain dynamic statistical parametric maps of cortical activity during semantic processing of visually presented words. An initial wave of activity was found to spread rapidly from occipital visual cortex to temporal, pariet al, and frontal areas within 185 ms, with a high degree of temporal overlap between different areas. Repetition effects were observed in many of the same areas following this initial wave of activation, providing evidence for the involvement of feedback mechanisms in repetition priming. We address the problem of learning structured unsupervised models with moment sparsity typical in many natural language induction tasks. For example, in unsupervised part-of-speech (POS) induction using hidden Markov models, we introduce a bias for words to be labeled by a small number of tags. In order to express this bias of posterior sparsity as opposed to parametric sparsity, we extend the posterior regularization framework [7]. We evaluate our methods on three languages — English, Bulgarian and Portuguese — showing consistent and significant accuracy improvement over EM-trained HMMs, and HMMs with sparsity-inducing Dirichlet priors trained by variational EM. We increase accuracy with respect to EM by 2.3 -6.5 in a purely unsupervised setting as well as in a weakly-supervised setting where the closed-class words are provided. Finally, we show improvements using our method when using the induced clusters as features of a discriminative model in a semi-supervised setting.