<jupyter_start><jupyter_text>Fake news  Data setwith real news too https://github.com/joolsa/fake_real_news_dataset Later maybe try http://www.fakenewschallenge.org/ https://github.com/FakeNewsChallenge/fnc-1 Later try polish and test on those stupid tweets But would model preserve some learning on this vs on a different language?  Dowload and get data into databuch<jupyter_code>path = Config.data_path()/'fakenews'
path.mkdir(parents=True, exist_ok=True)
# ! wget https://github.com/joolsa/fake_real_news_dataset/raw/master/fake_or_real_news.csv.zip -p {path}
! unzip {path}/fake_or_real_news.csv.zip -d {path}
path.ls()
csv_file = 'fake_or_real_news.csv'
df = pd.read_csv(path/csv_file)
df.head()
df['text'][0]
# df[['author','title', 'text']] = df[['author','title', 'text']].fillna('')
data_lm = (TextList.from_df(df, path, cols='text')
                .split_from_df(col=2)
                .label_from_df(cols='label')
                .databunch())
data_lm.show_batch()
data_lm.save()
data = load_data(path)<jupyter_output><empty_output><jupyter_text>Tokenization<jupyter_code>data = (TextList.from_df(df, path, cols='text')
                 .split_by_rand_pct(0.1)
                .label_from_df(cols='label')
                .databunch())
data.show_batch()
data.vocab.itos[:10]<jupyter_output><empty_output><jupyter_text>Language model<jupyter_code>bs = 48 // 4
data_lm = (TextList.from_df(df, path, cols='text')
                .split_by_rand_pct(0.1)
                .label_for_lm() # this is important !!!
                .databunch())
data_lm.save('data_lm.pkl')
data_lm = load_data(path, 'data_lm.pkl', bs=bs)
data_lm.show_batch()
learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.3)
learn.lr_find()
learn.recorder.plot(suggestion=True)
learn.fit_one_cycle(1, 6e-7, moms=(0.8,0.7))
learn.save('fit_head')
learn.load('fit_head')<jupyter_output>/opt/anaconda3/lib/python3.7/site-packages/fastai/basic_train.py:326: UserWarning: Wasn't able to properly load the optimizer state again.
  except: warn("Wasn't able to properly load the optimizer state again.")<jupyter_text>To compleate the finetuning, unfreeze and learn some more.<jupyter_code>learn.unfreeze()
learn.fit_one_cycle(10, 6e-6, moms=(0.8, 0.7)) # what are moms?
learn.recorder.plot_losses()
learn.save('fine_tuned')
TEXT = "I'm having good"
N_WORDS = 40
N_SENTENCES = 2
print("\n".join(learn.predict(TEXT, N_WORDS, temperature=0.75) for _ in range(N_SENTENCES)))
learn.save_encoder('finetuned_enc')<jupyter_output><empty_output><jupyter_text>Classifier<jupyter_code>data_clas = (TextList.from_df(df, path, cols='text')
                .split_by_rand_pct(0.1)
                .label_from_df(cols='label')
                .databunch())
data_clas.save('data_clas.pkl')
data_clas = load_data(path, 'data_clas.pkl', bs=bs)
data_clas.show_batch()
data_clas.vocab.itos = data_lm.vocab.itos # because of size mismatch error
learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.3)
learn.load_encoder('finetuned_enc')
learn.lr_find()
learn.recorder.plot(suggestion=True)
learn.fit_one_cycle(1, 2e-3, moms=(0.8, 0.7))
learn.save('first')
learn.load('first');
learn.freeze_to(-2)
learn.fit_one_cycle(1, slice(2e-3/(2.6**4), 2e-3), moms=(0.8,0.7))
learn.save('second')
learn.load('second')
learn.freeze_to(-3)
torch.cuda.memory_allocated() 
torch.cuda.empty_cache()
learn.fit_one_cycle(1, slice(5e-4/(2.6**4),5e-4), moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.save('third')
learn.load('third')
learn.unfreeze()
learn.fit_one_cycle(2, slice(2e-3/(2.6**4), 2e-3), moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.save('unfrozen')
learn.predict('BOSTON—Solving a problem that has long beset its customers, IKEA today opened its newest location next door to the South End Landfill, allowing customers to purchase their furniture and drop it off at the dump all in one convenient trip.')
learn.predict('AOC opposes Daylight Savings Time because “the extra hour of sunlight drastically speeds up climate change”')
learn.predict('“Private truck drivers are destroying America. They get like 4 miles to the gallon, they think they own the road and they throw bottles of pee out their windows so they can keep driving. Its a barbaric industry. Trucks should all be government regulated and drivers should be on full-time camera until we can figure out how to automate our shipping and do way with them altogether.')
# fake source https://www.snopes.com/fact-check/aoc-private-truck-drivers-quote/
learn.predict("""
     [O]n Independence Day, six Tempe police officers stopped by the Starbucks at Scottsdale Road and McKellips for coffee. The officers paid for their drinks and stood together having a cup of coffee before their long 4th of July shift. They were approached by a barista, who knew one of the officers by name, because he is a regular at that location. The barista said that a customer “did not feel safe” because of the police presence. The barista asked the officers to move out of the customer’s line of sight or to leave.

    Disappointed, the officers did in fact leave.

    This treatment of public safety workers could not be more disheartening. While the barista was polite, making such a request at all was offensive. Unfortunately, such treatment has become all too common in 2019.

    We know this is not a national policy at Starbucks Corporate and we look forward to working collaboratively with them on this important dialogue. 
""")
# this was actually true https://www.snopes.com/news/2019/07/06/police-reportedly-asked-to-leave-arizona-starbucks-because-customer-felt-unsafe/
learn.predict("""
     [F]rom late 2016 to early 2017, Starkey Water — the name of Whole Foods’ brand — recalled more than 2,000 cases of water after tests by regulators showed an impermissible level of arsenic beyond the federally mandated threshold of 10 parts per billion. A year later, Whole Foods’ internal testing showed results that were just under the federal limit but still at levels that pose risks if regularly consumed, according to growing research and independent experts, including Consumer Reports’ scientists.

    CR tracked down and reviewed hundreds of public records and test reports from bottled water brands, and from various federal and state regulators. We found that several popular brands sell bottled water with arsenic levels at or above 3 ppb; current research suggests that amounts above that level are potentially dangerous to drink over extended periods of time. CR believes the federal limit for bottled water should be revised to 3 ppb from the current federal standard of 10 ppb.

    In total, CR identified 11 brands out of more than 130 that either self-reported or, based on tests we commissioned, had detectable amounts of arsenic. Of those, six had levels of 3 ppb or higher. These brands are Starkey (owned by Whole Foods), Peñafiel (owned by Keurig Dr Pepper), Crystal Geyser Alpine Spring Water, Volvic (owned by Danone), and two regional brands, Crystal Creamery and EartH₂O.

    As part of our investigation, CR also was able to purchase two brands of imported water — Jermuk from Armenia and Peñafiel from Mexico — that are on an import alert issued by the federal government for previously having arsenic levels above the federal limit of 10 ppb. Such an alert is meant to “prevent potentially violative products from being distributed in the United States,” according to the Food and Drug Administration. Even so, CR easily purchased the two brands in retail stores in two states and on Amazon. 
""")
shutdown()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09723320158102766, 'fraction_numerical': 0.017127799736495388, 'mean_word_length': 4.720422004521477, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 5, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '611060', 'license': 'MIT', 'n_tokens_mistral': 2459, 'n_tokens_neox': 2322, 'n_words': 826, 'path': 'lesson3-own-fake-news.ipynb', 'repo_name': 'Mpreyzner/deep_learning_experiments', 'size': '7590'}