Emojis in Sentence Processing: An Electrophysiological Approach
This paper discusses two multi-experiment studies using the ERP methodology to investigate neural correlates of processing linguistic emojis. The first study examined the use of wink emojis used to mark irony and found the same ERP response complex that has been found in response to word-generated irony. Contingent upon individual differences in interpretation, these emojis are processed the same way as ironic words. The second study investigated the prediction of non-face emojis substituted for nouns. When predictability was high, unexpected emojis elicited the same ERP response patterns as words. Overall, the results of these two studies suggest that emojis used linguistically are processed in the same way as words and that individuals can integrate input from multiple modalities into a holistic representation of a single utterance.
Effect of emoji prime on the understanding of emotional words – evidence from ERPs
ABSTRACT Emojis play an increasingly important role as nonverbal symbols in online communication, and this study investigates their effect on the processing of subsequent words. We suppose that emojis, as an aid to emotional expression and semantic comprehension, prime the cognitive processing of the textual stimuli that follow. An affective priming experiment using ERPs was conducted, in which Semantic Relationship between the emoji and the word (Congruent, Incongruent, Non-related) was manipulated to assess their impact in an affective word categorization task. Behavioral results revealed no significant difference, while electrophysiological data showed otherwise. N1 component elicited by semantically congruent and incongruent emoji-word pairs showed greater amplitudes than those by non-related pairs. N400 elicited by incongruent pairs showed greater amplitudes than the N400 by congruent pairs. These data indicate that while emojis do not influence the outcomes of linguistic processing, emoji primes enhance attentional level, and semantically congruent contents demand fewer cognitive resources and facilitate better processing in comparison to incongruent ones. The findings point to emojis’ role not only as affective primes, but also as paralanguage that is semantically processed. Implications for computer-mediated communication are discussed.
Lexical versus conceptual anticipation during sentence processing: frontal positivity and N400 ERP components.
Although the sensitivity of the N400 to semantic processing is well established, late positive ERP components are also elicited during sentence comprehension. We suggest that there are multiple such components differing in scalp topography, and that a larger frontal positivity often follows the larger N400 elicited by congruent but unexpected sentence endings as compared to predictable endings. We evaluated the lexical versus conceptual specificity of this post-N400-positivity. High- and low-constraint sentences were completed by the words most preferred by a normative group (best completions), by words that were nearly synonymous to those best completions, and by other congruent words that were semantically dissimilar to the best completions. The N400 was sensitive to both the predictability (cloze probability) of the words and their semantic similarity to the best completion, consistent with a sensitivity to conceptual expectations that could be fulfilled by alternate words. In contrast, an anterior positivity was elicited by all final words that were not highly predictable, independent of whether they were semantically similar or dissimilar to the most preferred word, indicating a sensitivity to specific lexical expectations.