<jupyter_start><jupyter_text>0. PreparationYou have to move the API keys (in a .csv file) into the working folder (use the panel on the left)   1. TwitterAPI package<jupyter_code>!pip install TwitterAPI
from TwitterAPI import TwitterAPI
import pandas as pd
my_keys = pd.read_csv('TwitterAPIKeys.csv')

consumer_key = my_keys.iloc[0][1]
consumer_secret = my_keys.iloc[1][1]
api = TwitterAPI(consumer_key,
                 consumer_secret,
                 api_version='2', 
                 auth_type='oAuth2')
r = api.request('tweets/search/recent', {
        'query':'harrypotter',
        'tweet.fields':'author_id',
        'tweet.fields':'created_at',
        'expansions':'author_id',
        'max_results':100})
for item in r:
        print(item)<jupyter_output>{'created_at': '2022-03-04T11:22:46.000Z', 'author_id': '1357349686256955401', 'text': 'RT @wizardingwst_jp: みんなの声に反応する❗️❓\u3000\nこれを使えば魔法使いになった気分に🧙\u200d♂️⚡\n\U0001fa84音声認識\u3000魔法の杖\u3000ハリー・ポッター\U0001fa84\n#マホウドコロ オンラインで絶賛発売中✨\nhttps://t.co/zKHQUNWk2V\n\n#ハリポタ #ハリーポッタ…', 'id': '1499706915470807041'}
{'created_at': '2022-03-04T11:22:35.000Z', 'author_id': '8<PASSWORD>948', 'text': 'RT @Fnac_MusicaCine: Nuestras ofertas #HarryPotter llegan a su fin, pero antes, vamos a despedirlas...¡Con un último sorteo!🧙\u200d♀️\n\n✅Sigue a…', 'id': '1499706868431826946'}
{'created_at': '2022-03-04T11:21:59.000Z', 'author_id': '483846442', 'text': 'RT @wizardingwst_jp: あのスネイプ先生を大きく人型にかたどったクッションが登場🐍\n\n#マホウドコロ オンラインでは先行して受注予約を受付中です🌟お見逃しなく❗️\n\nhttps://t.co/eGt6q9xGKY\n※お渡しは4月頃を予定しています。\n\n＃ハリー…', 'id': '1499706717969727488'}
{'created_at': '2022-03-04T11:21:43.000Z', 'author_id': '931209739782053889', 'text': 'Vous aimez les #JeuxDeSociete et vivez dans les #HautsDeFrance alors ce #concours pour gagner #S[...]<jupyter_text>More details on the query arguments: https://developer.twitter.com/en/docs/twitter-api/tweets/search/api-reference/get-tweets-search-recent  2. TWARC<jupyter_code>!pip install twarc
!pip3 install --upgrade twarc-csv<jupyter_output>Collecting twarc
  Downloading twarc-2.9.4-py3-none-any.whl (55 kB)
[?25l
[K     |█████▉                          | 10 kB 26.3 MB/s eta 0:00:01
[K     |███████████▊                    | 20 kB 33.1 MB/s eta 0:00:01
[K     |█████████████████▋              | 30 kB 40.1 MB/s eta 0:00:01
[K     |███████████████████████▌        | 40 kB 42.9 MB/s eta 0:00:01
[K     |█████████████████████████████▍  | 51 kB 45.3 MB/s eta 0:00:01
[K     |████████████████████████████████| 55 kB 3.6 MB/s 
[?25hRequirement already satisfied: python-dateutil>=2.8 in /usr/local/lib/python3.7/dist-packages (from twarc) (2.8.2)
Requirement already satisfied: click<9,>=7 in /usr/local/lib/python3.7/dist-packages (from twarc) (7.1.2)
Requirement already satisfied: requests-oauthlib>=1.3 in /usr/local/lib/python3.7/dist-packages (from twarc) (1.3.1)
Collecting humanize>=3.9
  Downloading humanize-4.0.0-py3-none-any.whl (97 kB)
[?25l
[K     |███▍                            | 10 kB 40.7 MB/s eta 0:00:01
[K     |█[...]<jupyter_text>...then you will have to configure TWARC2 (version 2.0).  Run the cell, then add (manually) the "Bearer Token" (you find it in the csv file)  Then just select (n), i.e. no additional authentication methods Finally, please delete the output cell (because in it the BearerToken will be printed)<jupyter_code>!twarc2 configure
!twarc2 search --limit 100 "netflix" results.jsonl<jupyter_output>Set --limit of 100 reached:   0% 24677/604770000 [00:00<6:45:13, 24872.45it/s]<jupyter_text>More info on query arguments: https://twarc-project.readthedocs.io/en/latest/twarc2_en_us/<jupyter_code># convert the json into csv (friendlier for pandas)

!twarc2 csv results.jsonl tweets.csv
# read the csv and explore it

df = pd.read_csv('tweets.csv')
df.head()
for item in df.text:
  print(item)
for item in df.lang:
  print(item)<jupyter_output>en
en
in
en
ja
es
ja
th
in
en
en
in
ja
und
es
in
es
en
en
in
th
en
ro
ko
it
ja
in
in
ko
en
ko
pt
ja
es
en
es
und
in
in
en
en
ko
in
en
en
ja
es
ko
in
es
ko
in
es
in
pt
in
in
ja
es
in
in
es
tr
en
ja
in
en
th
in
pt
es
in
en
en
en
in
en
en
ko
hu
en
in
en
tl
ja
in
tr
ja
en
ja
in
en
tr
ja
en
en
in
es
ja
es
{'fraction_non_alphanumeric': 0.16283524904214558, 'fraction_numerical': 0.08884099616858238, 'mean_word_length': 3.840092699884125, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 18, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '126676', 'license': 'MIT', 'n_tokens_mistral': 2197, 'n_tokens_neox': 1897, 'n_words': 424, 'path': 'scripts/2.Scraping/Twitter_API_demo.ipynb', 'repo_name': 'BitMarta/DHLabVerona', 'size': '4171'}