This paper presents twAwler, a lightweight twitter crawler that targets language-specific communities of users. twAwler takes advantage of multiple endpoints of the twitter API to explore user relations and quickly recognize users belonging to the targetted set. It performs a complete crawl for all users, discovering many standard user relations, including the retweet graph, mention graph, reply graph, quote graph, follow graph, etc. twAwler respects all twitter policies and rate limits, while able to monitor large communities of active users. twAwler was used between August 2016 and March 2018 to generate an extensive dataset of close to all Greek-speaking twitter accounts (about 330 thousand) and their tweets and relations. In total, the crawler has gathered 750 million tweets of which 424 million are in Greek; 750 million follow relations; information about 300 thousand lists, their members (119 million member relations) and subscribers (27 thousand subscription relations); 705 thousand trending topics; information on 52 million users in total of which 292 thousand have been since suspended, 141 thousand have deleted their account, and 3.5 million are protected and cannot be crawled. twAwler mines the collected tweets for the retweet, quote, reply, and mention graphs, which, in addition to the follow relation crawled, offer vast opportunities for analysis and further research.
There are many settings in which users of a social media application provide evaluations of one another. In a variety of domains, mechanisms for evaluation allow one user to say whether he or she trusts another user, or likes the content they produced, or wants to confer special levels of authority or responsibility on them. Earlier work has studied how the relative status between two users - that is, their comparative levels of status in the group - affects the types of evaluations that one user gives to another. Here we study how similarity in the characteristics of two users can affect the evaluation one user provides of another. We analyze this issue under a range of natural similarity measures, showing how the interaction of similarity and status can produce strong effects. Among other consequences, we find that evaluations are less status-driven when users are more similar to each other; and we use effects based on similarity to provide a plausible mechanism for a complex phenomenon observed in studies of user evaluation, that evaluations are particularly low among users of roughly equal status. Our work has natural applications to the prediction of evaluation outcomes based on user characteristics, and the use of similarity information makes possible a novel application that we introduce here - to estimate the chance of a favorable overall evaluation from a group knowing only the attributes of the group's members, but not their expressed opinions. Behavioral economics tells us that emotions can profoundly affect individual behavior and decision-making. Does this also apply to societies at large, i.e. can societies experience mood states that affect their collective decision making? By extension is the public mood correlated or even predictive of economic indicators? Here we investigate whether measurements of collective mood states derived from large-scale Twitter feeds are correlated to the value of the Dow Jones Industrial Average (DJIA) over time. We analyze the text content of daily Twitter feeds by two mood tracking tools, namely OpinionFinder that measures positive vs. negative mood and Google-Profile of Mood States (GPOMS) that measures mood in terms of 6 dimensions (Calm, Alert, Sure, Vital, Kind, and Happy). We cross-validate the resulting mood time series by comparing their ability to detect the public's response to the presidential election and Thanksgiving day in 2008. A Granger causality analysis and a Self-Organizing Fuzzy Neural Network are then used to investigate the hypothesis that public mood states, as measured by the OpinionFinder and GPOMS mood time series, are predictive of changes in DJIA closing values. Our results indicate that the accuracy of DJIA predictions can be significantly improved by the inclusion of specific public mood dimensions but not others. We find an accuracy of 87.6 in predicting the daily up and down changes in the closing values of the DJIA and a reduction of the Mean Average Percentage Error by more than 6 . Index Terms—stock market prediction — twitter — mood analysis. Prediction of the popularity of online textual snippets gained much attention in recent years. In this paper we investigate some of the factors that contribute to popularity of specific phrases such as Twitter hashtags. We define a new prediction task and propose a linguistically motivated algorithm for accurate prediction of hashtag popularity. Our prediction algorithm successfully models the interplay between various constraints such as the length restriction, typing effort and ease of comprehension. Controlling for network structure and social aspects we get a glimpse into the processes that shape the way we produce language and coin new words. In order to learn the interactions between the constraints we cast the problem as a ranking task. We adapt Gradient Boosted Trees for learning ranking functions in order to predict the hashtags neologisms to be accepted. Our results outperform several baseline algorithms including SVM-rank, while maintaining higher interpretability, thus our model's prediction power can be used for better crafting of future hashtags. We study astroturf political campaigns on microblogging platforms: politically-motivated individuals and organizations that use multiple centrally-controlled accounts to create the appearance of widespread support for a candidate or opinion. We describe a machine learning framework that combines topological, content-based and crowdsourced features of information diffusion networks on Twitter to detect the early stages of viral spreading of political misinformation. We present promising preliminary results with better than 96 accuracy in the detection of astroturf content in the run-up to the 2010 U.S. midterm elections.