Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> Speakers and listeners make use of a variety of pragmatic factors to produce and identify sarcastic statements. It is also possible that lexical factors play a role, although this possibility has not been investigated previously. College students were asked to read excerpts from published works that originally contained the phrase said sarcastically, although the word sarcastically was deleted. The participants rated the characters' statements in these excerpts as more likely to be sarcastic than those from similar excerpts that did not originally contain the word sarcastically. The use of interjections, such as gee or gosh, predicted a significant amount of the variance in the participants' ratings of sarcastic intent. This outcome suggests that sarcastic statements may be more formulaic than previously realized. It also suggests that computer software could be written to recognize such lexical factors, greatly increasing the likelihood that non-literal intent could be correctly interpreted by such programs, even if they are unable to identify the pragmatic components of nonliteral language. <s> BIB001 </s> Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> Sarcasm is a sophisticated form of speech act widely used in online communities. Automatic recognition of sarcasm is, however, a novel task. Sarcasm recognition could contribute to the performance of review summarization and ranking systems. This paper presents SASI, a novel Semi-supervised Algorithm for Sarcasm Identification that recognizes sarcastic sentences in product reviews. SASI has two stages: semisupervised pattern acquisition, and sarcasm classification. We experimented on a data set of about 66000 Amazon reviews for various books and products. Using a gold standard in which each sentence was tagged by 3 annotators, we obtained precision of 77% and recall of 83.1% for identifying sarcastic sentences. We found some strong features that characterize sarcastic utterances. However, a combination of more subtle pattern-based features proved more promising in identifying the various facets of sarcasm. We also speculate on the motivation for using sarcasm in online communities and social networks. <s> BIB002 </s> Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> Deliberative, argumentative discourse is an important component of opinion formation, belief revision, and knowledge discovery; it is a cornerstone of modern civil society. Argumentation is productively studied in branches ranging from theoretical artificial intelligence to political rhetoric, but empirical analysis has suffered from a lack of freely available, unscripted argumentative dialogs. This paper presents the Internet Argument Corpus (IAC), a set of 390, 704 posts in 11, 800 discussions extracted from the online debate site 4forums.com. A 2866 thread/130, 206 post extract of the corpus has been manually sided for topic of discussion, and subsets of this topic-labeled extract have been annotated for several dialogic and argumentative markers: degrees of agreement with a previous post, cordiality, audiencedirection, combativeness, assertiveness, emotionality of argumentation, and sarcasm. As an application of this resource, the paper closes with a discussion of the relationship between discourse marker pragmatics, agreement, emotionality, and sarcasm in the IAC corpus. <s> BIB003 </s> Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> To avoid a sarcastic message being understood in its unintended literal meaning, in microtexts such as messages on Twitter.com sarcasm is often explicitly marked with the hashtag ‘#sarcasm’. We collected a training corpus of about 78 thousand Dutch tweets with this hashtag. Assuming that the human labeling is correct (annotation of a sample indicates that about 85% of these tweets are indeed sarcastic), we train a machine learning classifier on the harvested examples, and apply it to a test set of a day’s stream of 3.3 million Dutch tweets. Of the 135 explicitly marked tweets on this day, we detect 101 (75%) when we remove the hashtag. We annotate the top of the ranked list of tweets most likely to be sarcastic that do not have the explicit hashtag. 30% of the top-250 ranked tweets are indeed sarcastic. Analysis shows that sarcasm is often signalled by hyperbole, using intensifiers and exclamations; in contrast, non-hyperbolic sarcastic messages often receive an explicit marker. We hypothesize that explicit markers such as hashtags are the digital extralinguistic equivalent of nonverbal expressions that people employ in live interaction when conveying sarcasm. <s> BIB004 </s> Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> A common form of sarcasm on Twitter consists of a positive sentiment contrasted with a negative situation. For example, many sarcastic tweets include a positive sentiment, such as “love” or “enjoy”, followed by an expression that describes an undesirable activity or state (e.g., “taking exams” or “being ignored”). We have developed a sarcasm recognizer to identify this type of sarcasm in tweets. We present a novel bootstrapping algorithm that automatically learns lists of positive sentiment phrases and negative situation phrases from sarcastic tweets. We show that identifying contrasting contexts using the phrases learned through bootstrapping yields improved recall for sarcasm recognition. <s> BIB005 </s> Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> The relationship between context incongruity and sarcasm has been studied in linguistics. We present a computational system that harnesses context incongruity as a basis for sarcasm detection. Our statistical sarcasm classifiers incorporate two kinds of incongruity features: explicit and implicit. We show the benefit of our incongruity features for two text forms tweets and discussion forum posts. Our system also outperforms two past works (with Fscore improvement of 10-20%). We also show how our features can capture intersentential incongruity. <s> BIB006 </s> Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> The automatic detection of sarcasm and irony in user generated contents is one of the most challenging task of Natural Language Processing. In this paper we address this problem by introducing Bayesian Model Averaging (BMA), an ensemble approach to take into account several classifiers according to their reliabilities and their marginal probability predictions. The impact of the most used expressive signals (pragmatic particles and POS tags) have been evaluated in baseline models (traditional classifiers and majority voting) as well as in the proposed BMA approach. Experimental results highlight two main findings: (1) not all the features are equally able to characterize sarcasm and irony and (2) BMA not only outperforms traditional state of the art models, but is also able to ensure notable generalization capabilities both on ironic and sarcastic text. <s> BIB007 </s> Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> Precise semantic representation of a sentence and definitive information extraction are key steps in the accurate processing of sentence meaning, especially for figurative phenomena such as sarcasm, Irony, and metaphor cause literal meanings to be discounted and secondary or extended meanings to be intentionally profiled. Semantic modelling faces a new challenge in social media, because grammatical inaccuracy is commonplace yet many previous state-of-the-art methods exploit grammatical structure. For sarcasm detection over social media content, researchers so far have counted on Bag-of-Words(BOW), N-grams etc. In this paper, we propose a neural network semantic model for the task of sarcasm detection. We also review semantic modelling using Support Vector Machine (SVM) that employs constituency parsetrees fed and labeled with syntactic and semantic information. The proposed neural network model composed of Convolution Neural Network(CNN) and followed by a Long short term memory (LSTM) network and finally a Deep neural network(DNN). The proposed model outperforms state-of-the-art textbased methods for sarcasm detection, yielding an F-score of .92. <s> BIB008 </s> Automatic Sarcasm Detection: A Survey <s> Issues with Annotation <s> This paper is a novel study that views sarcasm detection in dialogue as a sequence labeling task, where a dialogue is made up of a sequence of utterances. We create a manuallylabeled dataset of dialogue from TV series ‘Friends’ annotated with sarcasm. Our goal is to predict sarcasm in each utterance, using sequential nature of a scene. We show performance gain using sequence labeling as compared to classification-based approaches. Our experiments are based on three sets of features, one is derived from information in our dataset, the other two are from past works. Two sequence labeling algorithms (SVM-HMM and SEARN) outperform three classification algorithms (SVM, Naive Bayes) for all these feature sets, with an increase in F-score of around 4%. Our observations highlight the viability of sequence labeling techniques for sarcasm detection of dialogue. <s> BIB009
Although hashtag-based labeling can provide large-scale supervision, the quality of the dataset may be dubious. This is particularly true in the case of using #not to indicate insincere sentiment. Reference BIB004 shows that #not is often used to express sarcasm-while the rest of the sentence is not sufficient for identifying the sarcasm. For example, "Looking forward to going back to school tomorrow #not." The speaker expresses sarcasm through #not. In most reported works that use hashtag-based supervision, the hashtag is removed in the pre-processing step. This reduces the sentence above to "Looking forward to going back to school tomorrow"-which may not have a sarcastic interpretation, unless the author's context is incorporated. Thus, hashtag-based supervision may cause ambiguities (or be incorrect) in some cases. To mitigate this problem, a new trend is to validate on multiple datasets-some annotated manually while others annotated through hashtags  BIB008 BIB006 . The authors of Reference BIB008 train their deep learning-based model using a large dataset of hashtag-annotated tweets but use a test set of manually annotated tweets. Even in the case of manually annotated datasets, the quality of the annotation is a concern. Since sarcasm is a subjective phenomenon, the inter-annotator agreement values reported in past work are diverse. Reference BIB002 indicates an agreement of 0.34. The value in the case of Reference BIB007 is 0.79, while for Reference BIB005 , it is 0.81. Reference  performs an interesting study on crosscultural sarcasm annotation. They compare annotations by Indian and American annotators and show that Indian annotators agree with each other more than their American counterparts. They also give examples to elicit these differences. For example, "It's sunny outside and I am at work. Yay" is considered sarcastic by the American annotators, but non-sarcastic by Indian annotators due to typical Indian climate. The above observations highlight the need of framing appropriate guidelines for annotators. Reference BIB003 presents an annotated dataset of discussion forums with many annotations (including sarcasm). They state that for the sarcasm annotation, the annotators answer the question: "Is the respondent using sarcasm?" The authors of Reference BIB001 describe their annotation guidelines in terms of three questions given to the annotators: (i) How likely is this excerpt to be sarcastic? (ii) How sure are you? (iii) Why do you think it is sarcastic? Their study then analyzes lexical indicators of sarcasm from the responses to the three questions. The authors of Reference BIB009 allow their annotators to look at a complete scene while annotating individual utterances. This allows them to understand the conversational context while annotating an utterance as sarcastic or not.