<jupyter_start><jupyter_text>Spooky Author IdentificationKaggle competition info found [here](https://www.kaggle.com/c/spooky-author-identification)  Getting the data Setting up the kaggle CLI<jupyter_code>! pip install kaggle --upgrade<jupyter_output>Collecting kaggle
[?25l  Downloading https://files.pythonhosted.org/packages/78/3a/64a6447e5faa313b70cb555e21b5a30718c95bcc4902d91784b57fbab737/kaggle-1.5.3.tar.gz (54kB)
[K    100% |████████████████████████████████| 61kB 37.6MB/s ta 0:00:01
[?25hRequirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /opt/anaconda3/lib/python3.7/site-packages (from kaggle) (1.24.1)
Requirement already satisfied, skipping upgrade: six>=1.10 in /opt/anaconda3/lib/python3.7/site-packages (from kaggle) (1.12.0)
Requirement already satisfied, skipping upgrade: certifi in /opt/anaconda3/lib/python3.7/site-packages (from kaggle) (2018.11.29)
Requirement already satisfied, skipping upgrade: python-dateutil in /opt/anaconda3/lib/python3.7/site-packages (from kaggle) (2.8.0)
Requirement already satisfied, skipping upgrade: requests in /opt/anaconda3/lib/python3.7/site-packages (from kaggle) (2.21.0)
Requirement already satisfied, skipping upgrade: tqdm in /opt/anaconda3/lib/python3.7/site-p[...]<jupyter_text>Then you need to upload your credentials from Kaggle on your instance. Login to kaggle and click on your profile picture on the top left corner, then 'My account'. Scroll down until you find a button named 'Create New API Token' and click on it. This will trigger the download of a file named 'kaggle.json'.Upload this file to the directory this notebook is running in, by clicking "Upload" on your main Jupyter page, then uncomment and execute the next two commands (or run them in a terminal). For Windows, uncomment the last two commands.<jupyter_code># ! mkdir -p ~/.kaggle/
# ! mv kaggle.json ~/.kaggle/
from fastai.text import *
Config.data_path()
path = Config.data_path()/'spooky-author'
path.mkdir(parents=True, exist_ok=True)
path
# ! kaggle competitions download -c spooky-author-identification -p {path}
path.ls()
# ! unzip /home/jupyter/.fastai/data/spooky-author/test.zip -d {path}
# ! unzip /home/jupyter/.fastai/data/spooky-author/train.zip -d {path}
# ! unzip /home/jupyter/.fastai/data/spooky-author/sample_submission.zip -d {path}
path.ls()
df_train = pd.read_csv(path/'train.csv')
df_train.head()
df_test = pd.read_csv(path/'test.csv')
df_test.head()
df_train['text'][0]
data_lm = TextDataBunch.from_csv(path, 'train.csv')
data_lm.save('lm-train')<jupyter_output><empty_output><jupyter_text>Next time we launch this notebook, we can skip the cell above that took a bit of time (and that will take a lot more when you get to the full dataset) and load those results like this:<jupyter_code>data = load_data(path, 'lm-train')
data = TextClasDataBunch.from_csv(path, 'train.csv')
data.show_batch()
df_train.columns
data = (TextList.from_csv(path, 'train.csv', cols='text')
           .random_split_by_pct(seed=42)
           .label_from_df(cols=2))
data
# Language model data
data_lm = TextLMDataBunch.from_csv(path, 'train.csv')
data_lm.save('data_lm_1.pkl')
# Classifier model data
data_clas = TextClasDataBunch.from_csv(path, 'train.csv', vocab=data_lm.train_ds.vocab, bs=32, label_cols='author')
data_clas.save('data_clas_1.pkl')
data_lm = load_data(path, fname='data_lm_1.pkl')
data_clas = load_data(path, fname='data_clas_1.pkl', bs=16)
learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.5)
learn.lr_find()
learn.recorder.plot()
learn.fit_one_cycle(1, 5e-2)
learn.recorder.plot_losses()
learn.predict('this is a test', n_words=10)
learn.unfreeze()
learn.fit_one_cycle(1, 1e-3)
learn.recorder.plot_losses()
learn.predict('this is a test', n_words=20)
learn.fit_one_cycle(1, 1e-4)
learn.recorder.plot_losses()
learn.predict('this is a test', n_words=20)
learn.save_encoder('ft_enc')
learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.5)
learn.load_encoder('ft_enc')
data_clas.show_batch()
learn.lr_find()
learn.recorder.plot()
learn.fit_one_cycle(1, 1e-2)
learn.freeze_to(-2)
learn.fit_one_cycle(1, slice(5e-3/2., 5e-3))
learn.predict('this is a test of ascent on the vice continent, and a somewhat distinct insurance of ground with Oxford.')
learn.metrics
learn.unfreeze()
learn.fit_one_cycle(1, slice(2e-3/100, 2e-3))
learn.recorder.plot_losses()
log(0.61)
sample_sub = pd.read_csv(path/'sample_submission.csv')
sample_sub.head()
len(df_test)
df_test.head(1)
df_test['text'][0]
learn.predict(df_test['text'][0])
learn.predict(df_test['text'][0])[0]
learn.save('class-predictor-1')
df_train.head()
learn.get_preds('test')
learn.predict(df_train['text'][0])
learn.predict(df_train['text'][1])
learn.predict(df_train['text'][3])
learn.predict(df_test['text'][1])
float(learn.predict(df_test['text'][1])[2][0])
preds = []
for i,row in df_test.iterrows():
    pred = learn.predict(row['text'])[2]
    [EAP, HPL, MWS] = [float(i) for i in pred]
    preds.append({ 'id': row['id'], 'EAP': EAP, 'HPL': HPL, 'MWS': MWS })
submission = pd.DataFrame(preds, columns=['id', 'EAP', 'HPL', 'MWS'])
submission.head()
len(submission)
submission.to_csv('/home/jupyter/tutorials/fastai/personal-learning/submission-1.csv', index=False)<jupyter_output><empty_output><jupyter_text>Try 2  Datablock API<jupyter_code>path.ls()
df_train.head()
df_test.head()
TextList.from_csv??
TextList.from_df??
bs = 64
data_lm = (TextList.from_csv(path, 'train.csv', cols='text')
              .random_split_by_pct(0.1, seed=42)
              .label_for_lm()
              .add_test(TextList.from_csv(path, 'test.csv', cols='text'))
              .databunch())
data_lm.save('data_lm.pkl')
data_lm = load_data(path, 'data_lm.pkl', bs=bs)
data_lm.show_batch()
learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.3)
learn.lr_find()
learn.recorder.plot(skip_end=10)
learn.fit_one_cycle(1, 3e-2, moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.save('fit_head.pkl')
learn.fit_one_cycle(1, 3e-3, moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.save('fit_head.pkl')
learn.freeze_to(-2)
learn.fit_one_cycle(1, slice(5e-3/2., 5e-3))
learn.predict('this is a test', n_words=10)
learn.predict('As I turned the', n_words=10)
learn.save('lm_fit_2.pkl')
learn.fit_one_cycle(10, 1e-3, moms=(0.8,0.7))
learn.recorder.plot_losses()
TEXT = "He began the process of"
N_WORDS = 40
N_SENTENCES = 2
print("\n".join(learn.predict(TEXT, N_WORDS, temperature=0.75) for _ in range(N_SENTENCES)))
learn.save('lm_fine_tuned_1.pkl')
learn.save_encoder('fine_tuned_enc.pkl')<jupyter_output><empty_output><jupyter_text>Classifier<jupyter_code>bs = 48
data_lm.train_ds.vocab
data_clas = (TextList.from_csv(path, 'train.csv', cols='text', vocab=data_lm.train_ds.vocab)
                .random_split_by_pct(0.1, seed=42)
                .label_from_df(cols='author')
                .add_test(TextList.from_csv(path, 'test.csv', cols='text'))
                .databunch())
data_clas.save('data_clas_2.pkl')
data_clas = load_data(path, 'data_clas_2.pkl', bs=bs)
data_clas.show_batch()
learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.5)
learn.load_encoder('fine_tuned_enc.pkl')
learn.lr_find()
learn.recorder.plot()
learn.fit_one_cycle(1, 5e-3, moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.save('clas_1.pkl')
learn.fit_one_cycle(1, 1e-3, moms=(0.8,0.7))
learn.fit_one_cycle(5, 1e-3, moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.freeze_to(-2)
learn.fit_one_cycle(1, slice(1e-3/(2.6**4),1e-2), moms=(0.8,0.7))
learn.fit_one_cycle(1, slice(1e-3/(2.6**4),1e-2), moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.save('clas_2.pkl')
learn.fit_one_cycle(3, slice(1e-3/(2.6**4),1e-2), moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.freeze_to(-3)
learn.fit_one_cycle(1, slice(5e-3/(2.6**4),5e-3), moms=(0.8,0.7))
learn.fit_one_cycle(2, slice(2e-3/(2.6**4),5e-3), moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.fit_one_cycle(3, slice(2e-3/(2.6**4),5e-3), moms=(0.8,0.7))
learn.save('clas_3.pkl')
learn.fit_one_cycle(3, slice(2e-3/(2.6**4),5e-3), moms=(0.8,0.7))
learn.save('clas_4.pkl')
learn.load('clas_4.pkl')
learn.data.batch_size
learn.unfreeze()
learn.fit_one_cycle(2, slice(1e-3/(2.6**4),1e-3), moms=(0.8,0.7))
learn.fit_one_cycle(5, slice(1e-3/(2.6**4),1e-3), moms=(0.8,0.7))
learn.save('clas_5.pkl')
learn.load('clas_5.pkl')
learn.recorder.plot_losses()
learn.fit_one_cycle(5, slice(1e-4/(2.6**4),1e-4), moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.fit_one_cycle(5, slice(1e-4/(2.6**4),1e-4), moms=(0.8,0.7))
learn.recorder.plot_losses()
learn.save('clas_6.pkl')
learn.export()
preds, targets = learn.get_preds(ds_type=DatasetType.Test)
preds[:10]
len(targets)
len(df_test)
df_test.head()
learn.data.classes
df_test.iloc[0]['id']<jupyter_output><empty_output><jupyter_text>Assuming that the ordering is the same in the preds and the original test csv<jupyter_code>formatted = []
for i,pred in enumerate(preds):
    [EAP, HPL, MWS] = [float(p) for p in pred]
    formatted.append({ 'id': df_test.iloc[i]['id'], 'EAP': EAP, 'HPL': HPL, 'MWS': MWS })
formatted[0]
submission = pd.DataFrame(formatted, columns=['id', 'EAP', 'HPL', 'MWS'])
submission.head()
submission.to_csv('submission-2.csv', index=False)
! kaggle competitions submit spooky-author-identification -f ./submission-2.csv -m "Attempt 2"<jupyter_output>100%|█████████████████████████████████████████| 562k/562k [00:04<00:00, 138kB/s]
Successfully submitted to Spooky Author Identification<jupyter_text>Something is messed up with my submission the rating is 3.04<jupyter_code>test = TextList.from_csv(path, 'test.csv', cols='text', vocab=data_clas.train_ds.vocab)
len(test)
learn = load_learner(path, test=test)
preds, targets = learn.get_preds(ds_type=DatasetType.Test)
preds[:10]
learn.data.classes
preds2 = []
for i,row in df_test.iterrows():
    pred = learn.predict(row['text'])[2]
    [EAP, HPL, MWS] = [float(i) for i in pred]
    preds2.append({ 'id': row['id'], 'EAP': EAP, 'HPL': HPL, 'MWS': MWS })
preds2[:5]
data_clas.train_ds.vocab
submission = pd.DataFrame(preds2, columns=['id', 'EAP', 'HPL', 'MWS'])
submission.head()
submission.to_csv('submission-2.csv', index=False)
! kaggle competitions submit spooky-author-identification -f ./submission-2.csv -m "Attempt 2 - formatting take 2"<jupyter_output>100%|█████████████████████████████████████████| 562k/562k [00:04<00:00, 127kB/s]
Successfully submitted to Spooky Author Identification
{'fraction_non_alphanumeric': 0.17841367390051854, 'fraction_numerical': 0.0451315536777415, 'mean_word_length': 5.861001317523057, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 28, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '573631', 'license': 'MIT', 'n_tokens_mistral': 4522, 'n_tokens_neox': 4310, 'n_words': 750, 'path': 'personal-learning/spooky-author/Spooky Author Identification.ipynb', 'repo_name': 'wdoug/misc-ml', 'size': '10414'}