Whether the online social media, like Twitter or its variant Weibo, can be a convincing proxy to predict the stock market has been debated for years, especially for China. However, as the traditional theory in behavioral finance states, the individual emotions can influence decision-making of investors, so it is reasonable to further explore this controversial topic from the perspective of online emotions, which is richly carried by massive tweets in social media. Surprisingly, through thorough study on over 10 million stock-relevant tweets from Weibo, both correlation analysis and causality test show that five attributes of the stock market in China can be competently predicted by various online emotions, like disgust, joy, sadness and fear. Specifically, the presented model significantly outperforms the baseline solutions on predicting five attributes of the stock market under the @math -means discretization. We also employ this model in the scenario of realistic online application and its performance is further testified.
In addition, there have long been controversies on predictive power of social media aiming at different fields @cite_14 @cite_0 . In the field of finance, found that public mood on Twitter could predict the Dow Jones Industrial Average @cite_3 . The public mood dimensions of Calm and Happiness seemed to have a predictive effect. However, the tweets they collected were associated with whole social status, not just the stock market in America, which could not represent online investors' sentiment. also showed stock micro-blog sentiments did have predictive power for market-adjusted returns. Instead of emotion on social media @cite_6 , some researchers examined textual representations in financial news articles for stock prediction @cite_2 @cite_16 . proposed a deep learning method for event-driven stock market prediction on large-scale financial news dataset @cite_7 . Besides, showed that daily trading volumes of stocks traded in NASDAQ100 were correlated with daily volumes of queries related to the same stocks @cite_4 .
@cite_14 @cite_4 @cite_7 @cite_6 @cite_3 @cite_0 @cite_2 @cite_16
2542080616 1971105953 2171468534 2037625889 2076386123 2041003138 1508206474 2043576616 2282408813 2078309899 359818833 2566234880 1489003673 2139328970 2250629460 1769818149 2057471594 2126267628 1996359281 1918540025
Predicting stock market movements is a well-known problem of interest. Now-a-days social media is perfectly representing the public sentiment and opinion about current events. Especially, Twitter has attracted a lot of attention from researchers for studying the public sentiments. Stock market prediction on the basis of public sentiments expressed on Twitter has been an intriguing field of research. Previous studies have concluded that the aggregate public mood collected from Twitter may well be correlated with Dow Jones Industrial Average Index (DJIA). The thesis of this work is to observe how well the changes in stock prices of a company, the rises and falls, are correlated with the public opinions being expressed in tweets about that company. Understanding author's opinion from a piece of text is the objective of sentiment analysis. The present paper have employed two different textual representations, Word2vec and N-gram, for analyzing the public sentiments in tweets. In this paper, we have applied sentiment analysis and supervised machine learning principles to the tweets extracted from Twitter and analyze the correlation between stock market movements of a company and sentiments in tweets. In an elaborate way, positive news and tweets in social media about a company would definitely encourage people to invest in the stocks of that company and as a result the stock price of that company would increase. At the end of the paper, it is shown that a strong correlation exists between the rise and falls in stock prices with the public sentiments in tweets. We live in a computerized and networked society where many of our actions leave a digital trace and affect other people’s actions. This has lead to the emergence of a new data-driven research field: mathematical methods of computer science, statistical physics and sociometry provide insights on a wide range of disciplines ranging from social science to human mobility. A recent important discovery is that search engine traffic (i.e., the number of requests submitted by users to search engines on the www) can be used to track and, in some cases, to anticipate the dynamics of social phenomena. Successful examples include unemployment levels, car and home sales, and epidemics spreading. Few recent works applied this approach to stock prices and market sentiment. However, it remains unclear if trends in financial markets can be anticipated by the collective wisdom of on-line users on the web. Here we show that daily trading volumes of stocks traded in NASDAQ-100 are correlated with daily volumes of queries related to the same stocks. In particular, query volumes anticipate in many cases peaks of trading by one day or more. Our analysis is carried out on a unique dataset of queries, submitted to an important web search engine, which enable us to investigate also the user behavior. We show that the query volume dynamics emerges from the collective but seemingly uncoordinated activity of many users. These findings contribute to the debate on the identification of early warnings of financial systemic risk, based on the activity of users of the www. Behavioral economics tells us that emotions can profoundly affect individual behavior and decision-making. Does this also apply to societies at large, i.e. can societies experience mood states that affect their collective decision making? By extension is the public mood correlated or even predictive of economic indicators? Here we investigate whether measurements of collective mood states derived from large-scale Twitter feeds are correlated to the value of the Dow Jones Industrial Average (DJIA) over time. We analyze the text content of daily Twitter feeds by two mood tracking tools, namely OpinionFinder that measures positive vs. negative mood and Google-Profile of Mood States (GPOMS) that measures mood in terms of 6 dimensions (Calm, Alert, Sure, Vital, Kind, and Happy). We cross-validate the resulting mood time series by comparing their ability to detect the public's response to the presidential election and Thanksgiving day in 2008. A Granger causality analysis and a Self-Organizing Fuzzy Neural Network are then used to investigate the hypothesis that public mood states, as measured by the OpinionFinder and GPOMS mood time series, are predictive of changes in DJIA closing values. Our results indicate that the accuracy of DJIA predictions can be significantly improved by the inclusion of specific public mood dimensions but not others. We find an accuracy of 87.6 in predicting the daily up and down changes in the closing values of the DJIA and a reduction of the Mean Average Percentage Error by more than 6 . Index Terms—stock market prediction — twitter — mood analysis. This paper describes early work trying to predict stock market indicators such as Dow Jones, NASDAQ and S&P 500 by analyzing Twitter posts. We collected the twitter feeds for six months and got a randomized subsample of about one hundredth of the full volume of all tweets. We measured collective hope and fear on each day and analyzed the correlation between these indices and the stock market indicators. We found that emotional tweet percentage significantly negatively correlated with Dow Jones, NASDAQ and S&P 500, but displayed significant positive correlation to VIX. It therefore seems that just checking on twitter for emotional outbursts of any kind gives a predictor of how the stock market will be doing the next day. Abstract Recent studies in behavioral finance discover that emotional impulses of stock investors affect stock prices. The challenge lies in how to quantify such sentiment to predict stock market movements. In this article, we propose a media-aware quantitative trading strategy utilizing sentiment information of Web media. This is achieved by capturing public mood from interactive behaviors of investors in social media and studying the impact of firm-specific news sentiment on stocks along with such public mood. Our experiments on the CSI 100 stocks during a three-month period show that a predictive performance in closeness to the actual future stock price is 0.612 in terms of root mean squared error, the same direction of price movement as the future price is 55.08 , and a simulation trading return is up to 166.11 . The increasing adoption of social media provides unprecedented opportunities to gain insight into human nature at vastly broader scales. Regarding the study of population-wide sentiment, prior research commonly focuses on text-based analyses and ignores a treasure trove of sentiment-laden content: images. In this paper, we make methodological and computational contributions by introducing the Smile Index as a formalized measure of societ al happiness. Detecting smiles in 9 million geo-located tweets over 16 months, we validate our Smile Index against both text-based techniques and self-reported happiness. We further make observational contributions by applying our metric to explore temporal trends in sentiment, relate public mood to societ al events, and predict economic indicators. Reflecting upon the innate, language-independent aspects of facial expressions, we recommend future improvements and applications to enable robust, global-level analyses. We conclude with implications for researchers studying and facilitating the expression of collective emotion through socio-technical systems. Our emotional state influences our choices. Research on how it happens usually comes from the lab. We know relatively little about how real world emotions affect real world settings, like financial markets. Here, we demonstrate that estimating emotions from weblogs provides novel information about future stock market prices. That is, it provides information not already apparent from market data. Specifically, we estimate anxiety, worry and fear from a dataset of over 20 million posts made on the site LiveJournal. Using a Granger-causal framework, we find that increases in expressions of anxiety, evidenced by computationally-identified linguistic features, predict downward pressure on the S&P 500 index. We also present a confirmation of this result via Monte Carlo simulation. The findings show how the mood of millions in a large online community, even one that primarily discusses daily life, can anticipate changes in a seemingly unrelated system. Beyond this, the results suggest new ways to gauge public opinion and predict its impact. We examined whether press reports on the collective mood of investors can predict changes in stock prices. We collected data on the use of emotion words in newspaper reports on traders' affect, coded these emotion words according to their location on an affective circumplex in terms of pleasantness and activation level, and created indices of collective mood for each trading day. Then, by using time series analyses, we examined whether these mood indices, depicting investors' emotion on a given trading day, could predict the next day's opening price of the stock market. The strongest findings showed that activated pleasant mood predicted increases in NASDAQ prices, while activated unpleasant mood predicted decreases in NASDAQ prices. We conclude that both valence and activation levels of collective mood are important in predicting trend continuation in stock prices. Previous studies have established the link between one's actions (e.g., engaging with others vs. minding one's own business) and one's national culture (e.g., collectivist vs. individualistic), and such actions have been shown to be important as they are collectively affiliated with a country's economic outcomes (e.g., Gross Domestic Product). Hitherto there has not been any systematic study of whether one's action on Twitter (e.g., deciding when to post messages) is linked to one's culture (e.g., country's Pace of Life). To fix that, we build different network snapshots starting from 55,000 seed users on Twitter, and we do so for 10 weeks across 30 countries (after filtering those with low penetration rates) for a total of 2.34 M profiles. Based on Hofstede's theory of cultural dimensions and Levine's Pace of Life theory, we consider three behavioral patterns on Twitter (i.e., temporal predictability of tweets, engaging with others, and supporting others who are less popular) and associate them with three different dimensions derived from the two theories: Pace of Life, Individualism and Power Distance. We find the following strong correlations: activity predictability negatively correlates with Pace of Life (r=-0.62), tweets with mentions negatively correlates with Individualism (r = -0.55), and power (e.g, Twitter popularity) imbalance in relationships (between, for example, two users mentioning each other) is correlated with Power Distance (r=0.62). These three cultural dimensions matter because they are associated with a country's socio-economic aspects - with GDP per capita, income inequality, and education expenditure. Studying the relationship between public sentiment and stock prices has been the focus of several studies. This paper analyzes whether the sentiment expressed in Twitter feeds, which discuss selected companies and their products, can indicate their stock price changes. To address this problem, an active learning approach was developed and applied to sentiment analysis of tweet streams in the stock market domain. The paper first presents a static Twitter data analysis problem, explored in order to determine the best Twitter-specific text preprocessing setting for training the Support Vector Machine (SVM) sentiment classifier. In the static setting, the Granger causality test shows that sentiments in stock-related tweets can be used as indicators of stock price movements a few days in advance, where improved results were achieved by adapting the SVM classifier to categorize Twitter posts into three sentiment categories of positive, negative and neutral (instead of positive and negative only). These findings were adopted in the development of a new stream-based active learning approach to sentiment analysis, applicable in incremental learning from continuously changing financial tweet streams. To this end, a series of experiments was conducted to determine the best querying strategy for active learning of the SVM classifier adapted to sentiment analysis of financial tweet streams. The experiments in analyzing stock market sentiments of a particular company show that changes in positive sentiment probability can be used as indicators of the changes in stock closing prices. We perform a sentiment analysis of all tweets published on the microblogging platform Twitter in the second half of 2008. We use a psychometric instrument to extract six mood states (tension, depression, anger, vigor, fatigue, confusion) from the aggregated Twitter content and compute a six-dimensional mood vector for each day in the timeline. We compare our results to a record of popular events gathered from media and sources. We find that events in the social, political, cultural and economic sphere do have a significant, immediate and highly specific effect on the various dimensions of public mood. We speculate that large scale analyses of mood can provide a solid platform to model collective emotive trends in terms of their predictive value with regards to existing social as well as economic indicators. Abstract In this paper, we propose a robust methodology to assess the value of microblogging data to forecast stock market variables: returns, volatility and trading volume of diverse indices and portfolios. The methodology uses sentiment and attention indicators extracted from microblogs (a large Twitter dataset is adopted) and survey indices (AAII and II, USMC and Sentix), diverse forms to daily aggregate these indicators, usage of a Kalman Filter to merge microblog and survey sources, a realistic rolling windows evaluation, several Machine Learning methods and the Diebold-Mariano test to validate if the sentiment and attention based predictions are valuable when compared with an autoregressive baseline. We found that Twitter sentiment and posting volume were relevant for the forecasting of returns of S&P 500 index, portfolios of lower market capitalization and some industries. Additionally, KF sentiment was informative for the forecasting of returns. Moreover, Twitter and KF sentiment indicators were useful for the prediction of some survey sentiment indicators. These results confirm the usefulness of microblogging data for financial expert systems, allowing to predict stock market behavior and providing a valuable alternative for existing survey measures with advantages (e.g., fast and cheap creation, daily frequency). The internet has made it feasible to tap a continuous stream of public sentiment from the world wide web, quite literally permitting one to "feel the pulse" of any issue under consideration. We present a methodology for real time sentiment extraction in the domain of finance. With the advent of the web, there has been a sharp increase in the influence of individuals on the stock market via web-based trading and the posting of sentiment to stock message boards. While it is importantto capture this "sentiment" of small investors, as yet, no index of sentiment has been compiled. This paper comprises (a) a technology for extracting small investor sentiment from web sources to create an index, and (b) illustrative applications of the methodology. We make use of computerized natural language and statistical algorithms for the automated classification of messages posted on the web. We design a suite of classification algorithms, each of different theoretical content, with a view to characterizing the sentiment of any single posting to a message board. The use of multiple methods allows imposition of voting rules in the classification process. It also enables elimination of "fuzzy" messages which are better off uninterpreted. A majority rule across algorithms vastly improves classification accuracy, but also leads to a natural increase in the number of messages classified as "fuzzy". The classifier achieves an accuracy of 62 (versus a random classification accuracy of 33 ), and compares favorably against human agreement on message classification, which was 72 . The technology is computationally efficient, allowing the access and interpretations of thousands of messages within minutes. Our illustrative applications show evidence of a strong link between market movements and sentiment. Based on approximately 25,000 messages for the last quarter of 2000, we found evidence that sentiment is based on stock movements. This study attempts to discover and evaluate the predictive power of stock micro blog sentiment on future stock price directional movements. We construct a set of robust models based on sentiment analysis and data mining algorithms. Using 72,221 micro blog postings for 1909 stock tickers and 3874 distinct authors, our study reveals not only that stock micro blog sentiments do have predictive power for simple and marketadjusted returns respectively, but also that this predictive accuracy is consistent with the underreaction hypothesis observed in behavioral finance. We establish that stock micro blog with its succinctness, high volume and real-time features do have predictive power over future stock price movements. Furthermore, this study provides support for the model of irrational investor sentiment, recommends a complimentary investing approach using user-generated content and validates an instrument that may contribute to the monetization schemes for Virtual Investing Communities. This paper proposes a technique to leverage topic based sentiments from Twitter to help predict the stock market. We first utilize a con- tinuous Dirichlet Process Mixture model to learn the daily topic set. Then, for each topic we derive its sentiment according to its opin- ion words distribution to build a sentiment time series. We then regress the stock index and the Twitter sentiment time series to predict the market. Experiments on real-life S&P100 Index show that our approach is effective and performs better than existing state-of-the-art non-topic based methods. Large-scale data from social media have a significant potential to describe complex phenomena in the real world and to anticipate collective behaviors such as information spreading and social trends. One specific case of study is represented by the collective attention to the action of political parties. Not surprisingly, researchers and stakeholders tried to correlate parties' presence on social media with their performances in elections. Despite the many efforts, results are still inconclusive since this kind of data is often very noisy and significant signals could be covered by (largely unknown) statistical fluctuations. In this paper we consider the number of tweets (tweet volume) of a party as a proxy of collective attention to the party, identify the dynamics of the volume, and show that this quantity has some information on the election outcome. We find that the distribution of the tweet volume for each party follows a log-normal distribution with a positive autocorrelation of the volume over short terms, which indicates the volume has large fluctuations of the log-normal distribution yet with a short-term tendency. Furthermore, by measuring the ratio of two consecutive daily tweet volumes, we find that the evolution of the daily volume of a party can be described by means of a geometric Brownian motion (i.e., the logarithm of the volume moves randomly with a trend). Finally, we determine the optimal period of averaging tweet volume for reducing fluctuations and extracting short-term tendencies. We conclude that the tweet volume is a good indicator of parties' success in the elections when considered over an optimal time window. Our study identifies the statistical nature of collective attention to political issues and sheds light on how to model the dynamics of collective attention in social media. Application of pattern recognition and machine learning in highly dynamic and data intensive financial markets is a popular research area amongst researchers and financial analysts. With evolving social dynamics of millions across the globe, it provides opportunity to make use of patterns in investor sentiment comprising of large scale microblog discussions to understand market movements and make an effective application in making hedging decisions. We apply sentiment analysis and machine learning principles to study causation between public collective sentiment and market movements. In this work we have used 0.6 million tweets for a period of November 2010 to June 2011, to run a practical simulation of hedging model for Dow Jones Industrial Average-DJIA Index. We have elaborated on how a simple hedging strategy like married-put can exercise use of weekly directional forecasts for DJIA to make portfolio adjustments from risky to high market conditions and vice versa. We have found the maximum of 91 SVM based binary classifier accuracy, towards direction (up and down prediction) estimations of DJIA. It has been shown that news events influence the trends of stock price movements. However, previous work on news-driven stock market prediction rely on shallow features (such as bags-of-words, named entities and noun phrases), which do not capture structured entity-relation information, and hence cannot represent complete and exact events. Recent advances in Open Information Extraction (Open IE) techniques enable the extraction of structured events from web-scale data. We propose to adapt Open IE technology for event-based stock price movement prediction, extracting structured events from large-scale public news without manual efforts. Both linear and nonlinear models are employed to empirically investigate the hidden and complex relationships between events and the stock market. Largescale experiments show that the accuracy of S&P 500 index prediction is 60 , and that of individual stock prediction can be over 70 . Our event-based system outperforms bags-of-words-based baselines, and previously reported systems trained on S&P 500 stock historical data. Human behaviour is highly individual by nature, yet statistical structures are emerging which seem to govern the actions of human beings collectively. Here we search for universal statistical laws dictating the timing of human actions in communication decisions. We focus on the distribution of the time interval between messages in human broadcast communication, as documented in Twitter, and study a collection of over 160,000 tweets for three user categories: personal (controlled by one person), managed (typically PR agency controlled) and bot-controlled (automated system). To test our hypothesis, we investigate whether it is possible to differentiate between user types based on tweet timing behaviour, independently of the content in messages. For this purpose, we developed a system to process a large amount of tweets for reality mining and implemented two simple probabilistic inference algorithms: 1. a naive Bayes classifier, which distinguishes between two and three account categories with classification performance of 84.6 and 75.8 , respectively and 2. a prediction algorithm to estimate the time of a user's next tweet with an . Our results show that we can reliably distinguish between the three user categories as well as predict the distribution of a user's inter-message time with reasonable accuracy. More importantly, we identify a characteristic power-law decrease in the tail of inter-message time distribution by human users which is different from that obtained for managed and automated accounts. This result is evidence of a universal law that permeates the timing of human decisions in broadcast communication and extends the findings of several previous studies of peer-to-peer communication. This paper covers design, implementation and evaluation of a system that may be used to predict future stock prices basing on analysis of data from social media services. The authors took advantage of large datasets available from Twitter micro blogging platform and widely available stock market records. Data was collected during three months and processed for further analysis. Machine learning was employed to conduct sentiment classification of data coming from social networks in order to estimate future stock prices. Calculations were performed in distributed environment according to Map Reduce programming model. Evaluation and discussion of results of predictions for different time intervals and input datasets proved efficiency of chosen approach is discussed here.