<jupyter_start><jupyter_text>In this notebook, I´m going to explain step by step what I did to get Trump´s twitter data from the first day of his presidency on January 20, 2017, until the 7th of February 2020.I´m going to use the tweepy API to extract the twitter data.- What: Trumps tweets- Time: from 07.01.2017 to 20.01.2020- How: tweepy AIP<jupyter_code># import the needed libraries
import os
import tweepy as tw
import pandas as pd<jupyter_output><empty_output><jupyter_text>The next step is to get a twitter developer account, this is needed to access the tweets.In previous research I created back than an account, therefore I use this access from the past.- consumer_key = "uYqdnmEfQdo8UCihFdYAKMST0"- consumer_secret = "<KEY>"- access_token = "<KEY>"- access_secret ="<KEY>"<jupyter_code>#Twitter credentials for the app
consumer_key= 'uYqdnmEfQdo8UCihFdYAKMST0'
consumer_secret= '<KEY>'
access_token= '<KEY>'
access_token_secret= '<KEY>'
#pass twitter credentials to tweepy
auth = tw.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tw.API(auth, wait_on_rate_limit=True)<jupyter_output><empty_output><jupyter_text>**Search tweet**<jupyter_code># testing the #realDonaldTrump
search_words = "@realDonaldTrump"
date_since = "2020-02-06"
# Collect tweets
tweets = tw.Cursor(api.search,
              q=search_words,
              lang="en",
              since=date_since).items(5)
tweets
type(tweets)
for tweet in tweets:
    print("------")
    print(tweet.text)
# Collect a list of tweets
[tweet.text for tweet in tweets]
new_search = search_words + " -filter:retweets"
tweets = tw.Cursor(api.search,
                   q=new_search,
                   lang="en",
                   since=date_since).items(5)

users_locs = [[tweet.user.screen_name, tweet.user.location, tweet.text] for tweet in tweets]
users_locs
# create a dataframe so save information
tweet_text = pd.DataFrame(data=users_locs, 
                    columns=['user', "location", "text"])
tweet_text
tweet_text.text[2]<jupyter_output><empty_output><jupyter_text>**Trump example**<jupyter_code>#make initial request for most recent tweets (200 is the maximum allowed count)
trumps_tweets = api.user_timeline(screen_name = "@realDonaldTrump",count=10, since_id = "2020-01-06")
trumps_tweets_list = [[tweet.user.screen_name, tweet.created_at, tweet.text] for tweet in trumps_tweets]
trumps_tweets_list

trumps_tweets_df = pd.DataFrame(data=trumps_tweets_list, 
                                columns=['user', "time", "text"])
trumps_tweets_df
trumps_tweets_df.text[0]<jupyter_output><empty_output><jupyter_text>**Problem definition**- Extract data for a specific **time period**- How to access the **full tweeted text** of the sentence- Decide if **retweet** should be used or not- Need to do **data cleaning**- Just **access** to tweets of the **last 7 days**  Problem: Full tweet text<jupyter_code>#make initial request 
trumps_tweets = api.user_timeline(screen_name = "@realDonaldTrump", count=10, 
                                  since_id = "2020-01-06",
                                  tweet_mode="extended")
# filter for specific attributes
trumps_tweets_list = [[tweet.user.screen_name, tweet.created_at, tweet.full_text] for tweet in trumps_tweets]

trumps_tweets_df = pd.DataFrame(data=trumps_tweets_list, 
                                columns=['user', "time", "text"])
trumps_tweets_df

trumps_tweets_df.text[1]<jupyter_output><empty_output><jupyter_text>Problem: retweetTrump tweets a lot (quantify) much, his own created tweets are most interesting.Other reasons why not retweets:- smaller amount of tweets- own tweets represent more important to him- uses twitter havily as communication plattform, retweets a looooottt!!- The question is how is emotions influences the US economy, especially his own tweets and retweets shoud have a bigger impact to this question **Quantification of his tweet activity**Therefor, check the frist 10 tweets and count the number of retweets.<jupyter_code>nr_of_retweets = 0
for tweet in trumps_tweets_df.text:
    word = tweet.split()
    if word[0] == "RT":
        nr_of_retweets = nr_of_retweets +1
print("Of the first X words are", nr_of_retweets, "retweets.")
end = trumps_tweets_df.time[0]
start = trumps_tweets_df.time[len(trumps_tweets_df.time)-1]
time_tweeted = end - start
print("Days:", time_tweeted.days)
print("Minutes:", round(time_tweeted.seconds/60))<jupyter_output>Days: 0
Minutes: 68<jupyter_text>Therefore I gonna use Tramp´s tweet data without his retweets. Create calculation with assumntion of how many tweets he posted in the 3 years.<jupyter_code># 10 tweet per hour * 8 hours a day * 365 for one year * 3 because he is president for 3 years
10 * 8 * 365 * 3<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09831285148927307, 'fraction_numerical': 0.019995834201208082, 'mean_word_length': 4.488, 'pattern_counts': {'":': 1, '<': 34, '<?xml version=': 0, '>': 34, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '499174', 'license': 'Apache-2.0', 'n_tokens_mistral': 1690, 'n_tokens_neox': 1552, 'n_words': 465, 'path': 'Data/tweepy/Get Twitter Data.ipynb', 'repo_name': 'Philhoels/Emotions_and_StockMarkets', 'size': '5061'}