This paper presents twAwler, a lightweight twitter crawler that targets language-specific communities of users. twAwler takes advantage of multiple endpoints of the twitter API to explore user relations and quickly recognize users belonging to the targetted set. It performs a complete crawl for all users, discovering many standard user relations, including the retweet graph, mention graph, reply graph, quote graph, follow graph, etc. twAwler respects all twitter policies and rate limits, while able to monitor large communities of active users. twAwler was used between August 2016 and March 2018 to generate an extensive dataset of close to all Greek-speaking twitter accounts (about 330 thousand) and their tweets and relations. In total, the crawler has gathered 750 million tweets of which 424 million are in Greek; 750 million follow relations; information about 300 thousand lists, their members (119 million member relations) and subscribers (27 thousand subscription relations); 705 thousand trending topics; information on 52 million users in total of which 292 thousand have been since suspended, 141 thousand have deleted their account, and 3.5 million are protected and cannot be crawled. twAwler mines the collected tweets for the retweet, quote, reply, and mention graphs, which, in addition to the follow relation crawled, offer vast opportunities for analysis and further research.
The study of user activity over time has been studied in previous work, showing that extraction of timeseries data from content may offer valuable insights into user behavior. mines several timeseries from user activity, including daily sentiment per entity or in total, idle time intervals, etc., facilitating further experimentation in that direction. Paraskevopoulos @cite_9 use the time series of twitter activity to correlate users without location information, with already geotagged users. Ferraz @cite_20 study inter-action time intervals in twitter activities and create a theoretical model that closely explains and reproduces observations. They use their model to discover outliers and detect bots. Bild @cite_13 focus on the Retweet Graph and reason about the effects of sampling on a set of metrics such as the distributions of tweets per user, tweet rates, and inter-event time intervals. They find that the Retweet Graph is small-world and scale-free similarly to the follow graph, but with stronger clustering.
@cite_9 @cite_13 @cite_20
1996359281 2404406225 2282408813 2065222371
Human behaviour is highly individual by nature, yet statistical structures are emerging which seem to govern the actions of human beings collectively. Here we search for universal statistical laws dictating the timing of human actions in communication decisions. We focus on the distribution of the time interval between messages in human broadcast communication, as documented in Twitter, and study a collection of over 160,000 tweets for three user categories: personal (controlled by one person), managed (typically PR agency controlled) and bot-controlled (automated system). To test our hypothesis, we investigate whether it is possible to differentiate between user types based on tweet timing behaviour, independently of the content in messages. For this purpose, we developed a system to process a large amount of tweets for reality mining and implemented two simple probabilistic inference algorithms: 1. a naive Bayes classifier, which distinguishes between two and three account categories with classification performance of 84.6 and 75.8 , respectively and 2. a prediction algorithm to estimate the time of a user's next tweet with an . Our results show that we can reliably distinguish between the three user categories as well as predict the distribution of a user's inter-message time with reasonable accuracy. More importantly, we identify a characteristic power-law decrease in the tail of inter-message time distribution by human users which is different from that obtained for managed and automated accounts. This result is evidence of a universal law that permeates the timing of human decisions in broadcast communication and extends the findings of several previous studies of peer-to-peer communication. Microblogging environments such as Twitter present a modality for interacting with information characterized by exposure to information “streams”. In this work, we examine what information in that stream is attended to, and how that attention corresponds to other aspects of microblog consumption and participation. To do this, we measured eye gaze, memory for content, interest ratings, and intended behavior of active Twitter users as they read their tweet steams. Our analyses focus on three sets of alignments: first, whether attention corresponds to other measures of user cognition such as memory (e.g., do people even remember what they attend to?); second, whether attention corresponds to behavior (e.g., are users likely to retweet content that is given the most attention); and third, whether attention corresponds to other attributes of the content and its presentation (e.g., do links attract attention?). We show a positive but imperfect alignment between user attention and other measures of user cognition like memory and interest, and between attention and behaviors like retweeting. To the third alignment, we show that the relationship between attention and attributes of tweets, such as whether it contains a link or is from a friend versus an organization, are complicated and in some cases counterintuitive. We discuss findings in relation to large scale phenomena like information diffusion and also suggest design directions to help maximize user attention in microblog environments. Previous studies have established the link between one's actions (e.g., engaging with others vs. minding one's own business) and one's national culture (e.g., collectivist vs. individualistic), and such actions have been shown to be important as they are collectively affiliated with a country's economic outcomes (e.g., Gross Domestic Product). Hitherto there has not been any systematic study of whether one's action on Twitter (e.g., deciding when to post messages) is linked to one's culture (e.g., country's Pace of Life). To fix that, we build different network snapshots starting from 55,000 seed users on Twitter, and we do so for 10 weeks across 30 countries (after filtering those with low penetration rates) for a total of 2.34 M profiles. Based on Hofstede's theory of cultural dimensions and Levine's Pace of Life theory, we consider three behavioral patterns on Twitter (i.e., temporal predictability of tweets, engaging with others, and supporting others who are less popular) and associate them with three different dimensions derived from the two theories: Pace of Life, Individualism and Power Distance. We find the following strong correlations: activity predictability negatively correlates with Pace of Life (r=-0.62), tweets with mentions negatively correlates with Individualism (r = -0.55), and power (e.g, Twitter popularity) imbalance in relationships (between, for example, two users mentioning each other) is correlated with Power Distance (r=0.62). These three cultural dimensions matter because they are associated with a country's socio-economic aspects - with GDP per capita, income inequality, and education expenditure. Twitter as one of the most important microblogging online social networks has attracted more than 200 million users in recent years. Although there have been several attempts on characterizing the Twitter by using incomplete sampled data, they have not been very successful to estimate the characteristics of the whole network. In this paper, we characterize Twitter by sampling from its social graph and user behaviors through a random walk based sampling technique called Respondent-Driven Sampling (RDS). To the best of our knowledge, for the first time RDS method and its estimator are used in order to obtain uniform unbiased estimation of several key structural and behavioral properties of Twitter. We compared the performance of the proposed method with other sampling methods such as Metropolis-Hasting Random Walk (MHRW) and sampling from active users (Timeline) against the uniform sampling (UNI). In order to gather the required data, we have implemented four independent crawlers. Our experimental results indicate that the RDS method exhibits lower estimation errors to the sample in- and out-degree distribution compared to MHRW and Timeline. We also show that RDS is more suitable to sample the followers vs. followings ratio, and the correlation between followers followings vs. tweets.