<jupyter_start><jupyter_text>I will regenerate these keys after submitting the task<jupyter_code>consumer_key = "Uo7d8jaq9moFWkMtamFm5rXCn"
consumer_secret = "<KEY>"
access_key = "<KEY>"
access_secret = "<KEY>"<jupyter_output><empty_output><jupyter_text>Authorization to consumer key and consumer secret<jupyter_code>auth = tweepy.OAuthHandler(consumer_key, consumer_secret)<jupyter_output><empty_output><jupyter_text>Access to user's access key and access secret<jupyter_code>auth.set_access_token(access_key, access_secret)<jupyter_output><empty_output><jupyter_text>Calling api<jupyter_code>api = tweepy.API(auth)<jupyter_output><empty_output><jupyter_text>a list to hold all the Tweets<jupyter_code>totaltweets = []<jupyter_output><empty_output><jupyter_text>I am using user_timeline method to extract the tweets. we can get only 200 tweets using user_timeline method.However, we can make successive requests for older tweets. The maximum number of tweets that we can get in a timeline is 3200.<jupyter_code>#makings initial requests for most recent 200 tweets
new_tweets = api.user_timeline(screen_name = "midasIIITD",count=200)
totaltweets.extend(new_tweets)<jupyter_output><empty_output><jupyter_text>saving the id of the oldest tweet less one<jupyter_code>oldest = totaltweets[-1].id - 1<jupyter_output><empty_output><jupyter_text>keep extracting tweets until there are no tweets left to extract<jupyter_code>while len(new_tweets) > 0:
    new_tweets = api.user_timeline(screen_name = "midasIIITD",count=200,max_id=oldest)
    #save most recent tweets
    totaltweets.extend(new_tweets)
    #update the id of the oldest tweet less one
    oldest = totaltweets[-1].id - 1
print(len(totaltweets))<jupyter_output>296<jupyter_text>Total 296 tweets get extracted Now dumping these tweets to a json file after collecting it to dictionary.<jupyter_code>temp={} 
for i,tweet in enumerate(totaltweets):
    temp[i]=tweet._json
with open('total_tweets.json', 'w') as fp:
    json.dump(temp, fp)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09198813056379822, 'fraction_numerical': 0.01533135509396637, 'mean_word_length': 7.834061135371179, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '538218', 'license': 'MIT', 'n_tokens_mistral': 753, 'n_tokens_neox': 704, 'n_words': 163, 'path': 'Python_Problem/tweets_part_1.ipynb', 'repo_name': 'ankur0rajput/MIDAS_TASK', 'size': '2152'}