Automatic Sarcasm Detection: A Survey <s> Short Text <s> Speakers and listeners make use of a variety of pragmatic factors to produce and identify sarcastic statements. It is also possible that lexical factors play a role, although this possibility has not been investigated previously. College students were asked to read excerpts from published works that originally contained the phrase said sarcastically, although the word sarcastically was deleted. The participants rated the characters' statements in these excerpts as more likely to be sarcastic than those from similar excerpts that did not originally contain the word sarcastically. The use of interjections, such as gee or gosh, predicted a significant amount of the variance in the participants' ratings of sarcastic intent. This outcome suggests that sarcastic statements may be more formulaic than previously realized. It also suggests that computer software could be written to recognize such lexical factors, greatly increasing the likelihood that non-literal intent could be correctly interpreted by such programs, even if they are unable to identify the pragmatic components of nonliteral language. <s> BIB001 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Irony is an effective but challenging mode of communication that allows a speaker to express sentiment-rich viewpoints with concision, sharpness and humour. Irony is especially common in online documents that express subjective and deeply-felt opinions, and thus represents a significant obstacle to the accurate analysis of sentiment in web texts. In this paper we look at one commonly used framing device for linguistic irony --the simile --to show how irony is often marked in ways that make it computationally feasible to detect. We conduct a very large corpus analysis of web-harvested similes to identify the most interesting characteristics of ironic comparisons, and provide an empirical evaluation of a new algorithm for separating ironic from non-ironic similes. <s> BIB002 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Sarcasm transforms the polarity of an apparently positive or negative utterance into its opposite. We report on a method for constructing a corpus of sarcastic Twitter messages in which determination of the sarcasm of each message has been made by its author. We use this reliable corpus to compare sarcastic utterances in Twitter to utterances that express positive or negative attitudes without sarcasm. We investigate the impact of lexical and pragmatic factors on machine learning effectiveness for identifying sarcastic utterances and we compare the performance of machine learning techniques and human judges on this task. Perhaps unsurprisingly, neither the human judges nor the machine learning techniques perform very well. <s> BIB003 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> The research described in this paper is focused on analyzing two playful domains of language: humor and irony, in order to identify key values components for their automatic processing. In particular, we are focused on describing a model for recognizing these phenomena in social media, such as ''tweets''. Our experiments are centered on five data sets retrieved from Twitter taking advantage of user-generated tags, such as ''#humor'' and ''#irony''. The model, which is based on textual features, is assessed on two dimensions: representativeness and relevance. The results, apart from providing some valuable insights into the creative and figurative usages of language, are positive regarding humor, and encouraging regarding irony. <s> BIB004 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> A common form of sarcasm on Twitter consists of a positive sentiment contrasted with a negative situation. For example, many sarcastic tweets include a positive sentiment, such as “love” or “enjoy”, followed by an expression that describes an undesirable activity or state (e.g., “taking exams” or “being ignored”). We have developed a sarcasm recognizer to identify this type of sarcasm in tweets. We present a novel bootstrapping algorithm that automatically learns lists of positive sentiment phrases and negative situation phrases from sarcastic tweets. We show that identifying contrasting contexts using the phrases learned through bootstrapping yields improved recall for sarcasm recognition. <s> BIB005 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> To avoid a sarcastic message being understood in its unintended literal meaning, in microtexts such as messages on Twitter.com sarcasm is often explicitly marked with the hashtag ‘#sarcasm’. We collected a training corpus of about 78 thousand Dutch tweets with this hashtag. Assuming that the human labeling is correct (annotation of a sample indicates that about 85% of these tweets are indeed sarcastic), we train a machine learning classifier on the harvested examples, and apply it to a test set of a day’s stream of 3.3 million Dutch tweets. Of the 135 explicitly marked tweets on this day, we detect 101 (75%) when we remove the hashtag. We annotate the top of the ranked list of tweets most likely to be sarcastic that do not have the explicit hashtag. 30% of the top-250 ranked tweets are indeed sarcastic. Analysis shows that sarcasm is often signalled by hyperbole, using intensifiers and exclamations; in contrast, non-hyperbolic sarcastic messages often receive an explicit marker. We hypothesize that explicit markers such as hashtags are the digital extralinguistic equivalent of nonverbal expressions that people employ in live interaction when conveying sarcasm. <s> BIB006 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Sarcasm is a common phenomenon in social media, and is inherently difficult to analyse, not just automatically but often for humans ::: too. It has an important effect on sentiment, but is usually ignored in social media analysis, because it is considered too tricky to handle. ::: While there exist a few systems which can detect sarcasm, almost no work has been carried out on studying the effect that sarcasm has on ::: sentiment in tweets, and on incorporating this into automatic tools for sentiment analysis. We perform an analysis of the effect of sarcasm ::: scope on the polarity of tweets, and have compiled a number of rules which enable us to improve the accuracy of sentiment analysis when ::: sarcasm is known to be present. We consider in particular the effect of sentiment and sarcasm contained in hashtags, and have developed ::: a hashtag tokeniser for GATE, so that sentiment and sarcasm found within hashtags can be detected more easily. According to our experiments, ::: the hashtag tokenisation achieves 98% Precision, while the sarcasm detection achieved 91% Precision and polarity detection 80%. <s> BIB007 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Automatically detecting verbal irony (roughly, sarcasm) is a challenging task because ironists say something other than ‐ and often opposite to ‐ what they actually mean. Discerning ironic intent exclusively from the words and syntax comprising texts (e.g., tweets, forum posts) is therefore not always possible: additional contextual information about the speaker and/or the topic at hand is often necessary. We introduce a new corpus that provides empirical evidence for this claim. We show that annotators frequently require context to make judgements concerning ironic intent, and that machine learning approaches tend to misclassify those same comments for which annotators required additional context. <s> BIB008 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Sarcasm is generally characterized as a figure of speech that involves the substitution of a literal by a figurative meaning, which is usually the opposite of the original literal meaning. We re-frame the sarcasm detection task as a type of word sense disambiguation problem, where the sense of a word is either literal or sarcastic. We call this the Literal/Sarcastic Sense Disambiguation (LSSD) task. We address two issues: 1) how to collect a set of target words that can have either literal or sarcastic meanings depending on context; and 2) given an utterance and a target word, how to automatically detect whether the target word is used in the literal or the sarcastic sense. For the latter, we investigate several distributional semantics methods and show that a Support Vector Machines (SVM) classifier with a modified kernel using word embeddings achieves a 7-10% F1 improvement over a strong lexical baseline. <s> BIB009 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> This report summarizes the objectives and evaluation of the SemEval 2015 task on the sentiment analysis of figurative language on Twitter (Task 11). This is the first sentiment analysis task wholly dedicated to analyzing figurative language on Twitter. Specifically, three broad classes of figurative language are considered: irony, sarcasm and metaphor. Gold standard sets of 8000 training tweets and 4000 test tweets were annotated using workers on the crowdsourcing platform CrowdFlower. Participating systems were required to provide a fine-grained sentiment score on an 11-point scale (-5 to +5, including 0 for neutral intent) for each tweet, and systems were evaluated against the gold standard using both a Cosinesimilarity and a Mean-Squared-Error measure. <s> BIB010 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Sentiment Analysis is a technique to identify people's opinion, attitude, sentiment, and emotion towards any specific target such as individuals, events, topics, product, organizations, services etc. Sarcasm is a special kind of sentiment that comprise of words which mean the opposite of what you really want to say (especially in order to insult or wit someone, to show irritation, or to be funny). People often expressed it verbally through the use of heavy tonal stress and certain gestural clues like rolling of the eyes. These tonal and gestural clues are obviously not available for expressing sarcasm in text, making its detection reliant upon other factors. In this paper, two approaches to detect sarcasm in the text of Twitter data were proposed. The first is a parsing-based lexicon generation algorithm (PBLGA) and the second was to detect sarcasm based on the occurrence of the interjection word. The combination of two approaches is also shown and compared with the existing state-of-the-art approach to detect sarcasm. First approach attains a 0.89, 0.81 and 0.84 precision, recall and f -- score respectively. Second approach attains 0.85, 0.96 and 0.90 precision, recall and f -- score respectively in tweets with sarcastic hashtag. <s> BIB011 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Opinion mining and sentiment analysis refer to the identification and the aggregation of attitudes or opinions expressed by internet users towards a specific topic. However, due to the limitation in terms of characters (i.e. 140 characters per tweet) and the use of informal language, the state-of-the-art approaches of sentiment analysis present lower performances in Twitter than that when they are applied on longer texts. Moreover, presence of sarcasm makes the task even more challenging. Sarcasm is when a person conveys implicit information, usually the opposite of what is said, within the message he transmits. In this paper we propose a method that makes use of a minimal set of features, yet, efficiently classifies tweets regardless of their topic. We also study the importance of detecting sarcastic tweets automatically, and demonstrate how the accuracy of sentiment analysis can be enhanced knowing which tweets are sarcastic and which are not. <s> BIB012 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> The relationship between context incongruity and sarcasm has been studied in linguistics. We present a computational system that harnesses context incongruity as a basis for sarcasm detection. Our statistical sarcasm classifiers incorporate two kinds of incongruity features: explicit and implicit. We show the benefit of our incongruity features for two text forms tweets and discussion forum posts. Our system also outperforms two past works (with Fscore improvement of 10-20%). We also show how our features can capture intersentential incongruity. <s> BIB013 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Sarcasm requires some shared knowledge between speaker and audience; it is a profoundly contextual phenomenon. Most computational approaches to sarcasm detection, however, treat it as a purely linguistic matter, using information such as lexical cues and their corresponding sentiment as predictive features. We show that by including extra-linguistic information from the context of an utterance on Twitter — such as properties of the author, the audience and the immediate communicative environment — we are able to achieve gains in accuracy compared to purely linguistic features in the detection of this complex phenomenon, while also shedding light on features of interpersonal interaction that enable sarcasm in conversation. <s> BIB014 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> The automatic detection of sarcasm and irony in user generated contents is one of the most challenging task of Natural Language Processing. In this paper we address this problem by introducing Bayesian Model Averaging (BMA), an ensemble approach to take into account several classifiers according to their reliabilities and their marginal probability predictions. The impact of the most used expressive signals (pragmatic particles and POS tags) have been evaluated in baseline models (traditional classifiers and majority voting) as well as in the proposed BMA approach. Experimental results highlight two main findings: (1) not all the features are equally able to characterize sarcasm and irony and (2) BMA not only outperforms traditional state of the art models, but is also able to ensure notable generalization capabilities both on ironic and sarcastic text. <s> BIB015 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Sarcasm understanding may require information beyond the text itself, as in the case of ‘I absolutely love this restaurant!’ which may be sarcastic, depending on the contextual situation. We present the first quantitative evidence to show that historical tweets by an author can provide additional context for sarcasm detection. Our sarcasm detection approach uses two components: a contrast-based predictor (that identifies if there is a sentiment contrast within a target tweet), and a historical tweet-based predictor (that identifies if the sentiment expressed towards an entity in the target tweet agrees with sentiment expressed by the author towards that entity in the past). <s> BIB016 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Sarcasm is a nuanced form of language in which individuals state the opposite of what is implied. With this intentional ambiguity, sarcasm detection has always been a challenging task, even for humans. Current approaches to automatic sarcasm detection rely primarily on lexical and linguistic cues. This paper aims to address the difficult task of sarcasm detection on Twitter by leveraging behavioral traits intrinsic to users expressing sarcasm. We identify such traits using the user's past tweets. We employ theories from behavioral and psychological studies to construct a behavioral modeling framework tuned for detecting sarcasm. We evaluate our framework and demonstrate its efficiency in identifying sarcastic tweets. <s> BIB017 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Automatically detecting verbal irony (roughly, sarcasm) in online content is important for many practical applications (e.g., sentiment detection), but it is difficult. Previous approaches have relied predominantly on signal gleaned from word counts and grammatical cues. But such approaches fail to exploit the context in which comments are embedded. We thus propose a novel strategy for verbal irony classification that exploits contextual features, specifically by combining noun phrases and sentiment extracted from comments with the forum type (e.g., conservative or liberal) to which they were posted. We show that this approach improves verbal irony classification performance. Furthermore, because this method generates a very large feature space (and we expect predictive contextual features to be strong but few), we propose a mixed regularization strategy that places a sparsity-inducing `1 penalty on the contextual feature weights on top of the `2 penalty applied to all model coefficients. This increases model sparsity and reduces the variance of model performance. <s> BIB018 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> Sarcasm can radically alter or invert a phrase’s meaning. Sarcasm detection can therefore help improve natural language processing (NLP) tasks. The majority of prior research has modeled sarcasm detection as classification, with two important limitations: 1. Balanced datasets, when sarcasm is actually rather rare. 2. Using Twitter users’ self-declarations in the form of hashtags to label data, when sarcasm can take many forms. To address these issues, we create an unbalanced corpus of manually annotated Twitter conversations. We compare human and machine ability to recognize sarcasm on this data under varying amounts of context. Our results indicate that both class imbalance and labelling method affect performance, and should both be considered when designing automatic sarcasm detection systems. We conclude that for progress to be made in real-world sarcasm detection, we will require a new class labelling scheme that is able to access the ‘common ground’ held between conversational parties. <s> BIB019 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> In this paper, we propose a novel mechanism for enriching the feature vector, for the task of sarcasm detection, with cognitive features extracted from eye-movement patterns of human readers. Sarcasm detection has been a challenging research problem, and its importance for NLP applications such as review summarization, dialog systems and sentiment analysis is well recognized. Sarcasm can often be traced to incongruity that becomes apparent as the full sentence unfolds. This presence of incongruity- implicit or explicit- affects the way readers eyes move through the text. We observe the difference in the behaviour of the eye, while reading sarcastic and non sarcastic sentences. Motivated by this observation, we augment traditional linguistic and stylistic features for sarcasm detection with the cognitive features obtained from readers eye movement data. We perform statistical classification using the enhanced feature set so obtained. The augmented cognitive features improve sarcasm detection by 3.7% (in terms of Fscore), over the performance of the best reported system. <s> BIB020 </s> Automatic Sarcasm Detection: A Survey <s> Short Text <s> We introduce the Self-Annotated Reddit Corpus (SARC), a large corpus for sarcasm research and for training and evaluating systems for sarcasm detection. The corpus has 1.3 million sarcastic statements -- 10 times more than any previous dataset -- and many times more instances of non-sarcastic statements, allowing for learning in both balanced and unbalanced label regimes. Each statement is furthermore self-annotated -- sarcasm is labeled by the author, not an independent annotator -- and provided with user, topic, and conversation context. We evaluate the corpus for accuracy, construct benchmarks for sarcasm detection, and evaluate baseline methods. <s> BIB021
Social media makes large-scale user-generated text accessible. However, because of restrictions on text length imposed by some of these platforms, this text tends to be short requiring authors to use abbreviations to fit their statements within the specific limit. Despite this noise, datasets of tweets have been popular for sarcasm detection because of availability of the Twitter API and popularity of Twitter as a medium. For Twitter-based datasets, two approaches to obtain annotations have been used. The first is manual annotation. Reference BIB005 introduced a dataset of tweets that were manually annotated as either sarcastic or not. Reference BIB007 studied sarcastic tweets and their impact to sentiment classification. They experimented with around 600 tweets that were marked for subjectivity, sentiment, and sarcasm. The second technique to create datasets is the use of hashtag-based supervision. Many approaches use hashtags in tweets as indicators of sarcasm, to create labeled datasets. The popularity of this approach (over manual annotation) can be attributed to various factors: (1) Nobody but the author of a tweet can determine with certainty whether the tweet was intended to be sarcastic or not. A hashtag is a label provided by authors themselves. BIB009  BIB001 BIB020 BIB002 (2) This approach allows rapid creation of large-scale datasets, since manual effort is restricted. To create such a dataset, tweets containing particular hashtags are labeled as sarcastic. Reference  uses a dataset of tweets that are labeled based on the presence of sarcasmindicative hashtags such as #sarcasm, #sarcastic, #not, and so on. Reference BIB003 also uses hashtagbased supervision for tweets. However, they retain only examples where it occurs at the end of a tweet but eliminate cases where the hashtag is a part of the running text. For example, "#sarcasm is popular among teens" is eliminated, while "I got a parking fine on my birthday. #amazing #bestbirthdayever #sarcasm" is included. Reference BIB004 uses a similar approach. Reference  presents a large dataset of 40,000 tweets labeled as sarcastic or not, again using hashtags. Reference BIB010 presents hashtag-annotated dataset of tweets that they divide into three parts: 1,000 trial; 4,000 development; and 8,000 test tweets. Reference BIB006 uses "#not" to download and label their tweets. Reference  creates a dataset using hashtag-based supervision based on hashtags indicated by multiple labels, namely, politics, education, sarcasm, humor, and irony. Note that although these labels are not mutually exclusive by definition (for example, sarcasm is a form of irony), the presence of hashtags is used as a sole indicator of a certain label. Therefore, a tweet may be humorous and about politics, but this approach labels it purely on the basis of the hashtag present in it. For example, "Politicians are never wrong #politics" would get labeled as a political tweet, while "Politicians are never wrong #sarcasm" would get labeled as a sarcastic tweet. Other works using this approach have also been reported BIB019  BIB011 BIB012 BIB013 . The Twitter API requires you to search for a keyword in a tweet (in addition to other features that it provides). Also, an author indicating, through a hashtag, that they are being sarcastic is straightforward. How to collect non-sarcastic tweets becomes tricky. Tweets containing "#notsarcastic" may be downloaded as non-sarcastic but are unlikely to have the statistical properties of general non-sarcastic text. In some cases, authors have assumed that tweets not containing the "#sarcastic" hashtag are non-sarcastic BIB014 . Another option is to create a list of authors of sarcastic tweets and download their tweets not containing the sarcasm-indicative hashtag. Hashtag-based supervision may lead to a degradation in the quality of the training data for reasons such as incorrect use of a sarcasm-indicative hashtag. To ensure quality, Reference BIB015 obtains the initial label based on the presence of hashtags, following which these labels are manually verified and corrected by annotators. Twitter also provides access to additional context in terms of past tweets or author information. Hence, to predict sarcasm, supplementary datasets have also been used for sarcasm detection. "Supplementary" datasets refer to text that does not need to be annotated but that will contribute to the judgment of the sarcasm detector. Reference BIB016 uses a supplementary set of complete Twitter timelines (limited to 3,200 tweets, by Twitter) to establish context for a given dataset of tweets. Reference BIB017 uses a dataset of tweets, labeled by hashtag-based supervision along with a historical context of 80 tweets per author. Like supplementary datasets, supplementary annotation (i.e., annotation apart from sarcasm/ non-sarcasm) has also been explored. Reference BIB020 captures cognitive features based on eyetracking. They employ annotators who are asked to determine the sentiment (and not "sarcasm/ not-sarcasm," since, as per their claim, it can result in priming) of a text. While the annotators read the text, their eye movements are recorded by an eye-tracker. This eye-tracking information serves as supplementary annotation. Other social media text used for sarcasm-labeled datasets includes posts from Reddit. BIB012 Reference BIB008 creates a corpus of Reddit posts of 10K sentences, from six Reddit topics. Reference BIB018 presents a dataset of Reddit comments-5,625 sentences. Similarly, Reference BIB021 presents a large dataset of manually labeled reddit comments, including 1.3 million sarcastic comments.