<jupyter_start><jupyter_text>CS 72 Accelerated Computational Linguistics Final Project  Training the Baseline Model Credits:* BiDAF code and data provided by: * GPU provided Google Co-lab:  * Google Colab notebook written and model trained by: <NAME> '21 (<EMAIL>)  Bi-directional Attention Flow (BiDAF) model  Image Credits:    First we import the code into Google Colab by cloning the git repository<jupyter_code>!git clone https://github.com/Yakoob-Khan/squad<jupyter_output>Cloning into 'squad'...
remote: Enumerating objects: 53, done.[K
remote: Counting objects: 100% (53/53), done.[K
remote: Compressing objects: 100% (46/46), done.[K
remote: Total 97 (delta 16), reused 40 (delta 7), pack-reused 44[K
Unpacking objects: 100% (97/97), done.<jupyter_text>Now we change into the directory containing code for the baseline model<jupyter_code>%cd /content/squad<jupyter_output>[Errno 2] No such file or directory: '/content/squad'
/Users/yakoobkhan/Desktop/CS-72/Project/squad/baseline-model<jupyter_text>We need to get and install the Anaconda Package as Google Colab does not have it by default.<jupyter_code>!wget -c https://repo.continuum.io/archive/Anaconda3-5.3.0-Linux-x86_64.sh

!chmod +x Anaconda3-5.3.0-Linux-x86_64.sh

!bash ./Anaconda3-5.3.0-Linux-x86_64.sh -b -f -p /usr/local

import sys

sys.path.append('/usr/local/lib/python3.6/site-packages/')
!conda --version<jupyter_output>conda 4.8.3<jupyter_text>Update conda environment to the latest version.<jupyter_code>!conda update -n base conda
!conda --version<jupyter_output>conda 4.8.3<jupyter_text>Run environment.yml to create the "squad" conda environment.<jupyter_code>!conda env create -f environment.yml<jupyter_output>Collecting package metadata (repodata.json): - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done
Solving environment: / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \[...]<jupyter_text>Check that the squad environment has been correctly installed.<jupyter_code>!conda info --envs<jupyter_output># conda environments:
#
base                  *  /usr/local
squad                    /usr/local/envs/squad<jupyter_text>We need to change the bash.rc file so that we can run a bash shell inside a jupyter notebook cell<jupyter_code>!echo ". /usr/local/etc/profile.d/conda.sh" >> ~/.bashrc<jupyter_output><empty_output><jupyter_text>Run "python setup.py" to download the GloVe word embeddings and pre-process the training and dev datasets. This process takes about 30 minutes.<jupyter_code>!bash -i<jupyter_output>bash: cannot set terminal process group (2595): Inappropriate ioctl for device
bash: no job control in this shell
]0;root@1fa96ba23b1b: /content[01;32mroot@1fa96ba23b1b[00m:[01;34m/content[00m# conda activate squad
(squad) ]0;root@1fa96ba23b1b: /content[01;32mroot@1fa96ba23b1b[00m:[01;34m/content[00m# python setup.py
python: can't open file 'setup.py': [Errno 2] No such file or directory
(squad) ]0;root@1fa96ba23b1b: /content[01;32mroot@1fa96ba23b1b[00m:[01;34m/content[00m# ls
[0m[01;34msample_data[0m  [01;34msquad[0m
(squad) ]0;root@1fa96ba23b1b: /content[01;32mroot@1fa96ba23b1b[00m:[01;34m/content[00m# cd squad
(squad) ]0;root@1fa96ba23b1b: /content/squad[01;32mroot@1fa96ba23b1b[00m:[01;34m/content/squad[00m# ls
 [0m[01;32mAnaconda3-5.3.0-Linux-x86_64.sh[0m   [01;34mdata[0m              models.py   test.py
 args.py                           environment.yml   README.md   train.py
'Code Experiments.ipynb'           layers.py         [01;34msave[...]<jupyter_text>We are ready to train the BiDAF model! Training takes about 10 minutes per epoch on a single GPU on Google Colab. We trained the model for 14 epochs for a total training time of about 2.5 hours. * DEV NLL is the development / validation set training loss.  * F1 is the F1 Score. * EM is the Exact Match Score * Scroll all the way to the bottom to see the final F1 and EM score!<jupyter_code>!bash -i<jupyter_output>bash: cannot set terminal process group (2595): Inappropriate ioctl for device
bash: no job control in this shell
]0;root@1fa96ba23b1b: /content[01;32mroot@1fa96ba23b1b[00m:[01;34m/content[00m# conda activate squad
(squad) ]0;root@1fa96ba23b1b: /content[01;32mroot@1fa96ba23b1b[00m:[01;34m/content[00m# ls
[0m[01;34msample_data[0m  [01;34msquad[0m
alse
python: can't open file 'train.py': [Errno 2] No such file or directory
(squad) ]0;root@1fa96ba23b1b: /content[01;32mroot@1fa96ba23b1b[00m:[01;34m/content[00m# cd squad
(squad) ]0;root@1fa96ba23b1b: /content/squad[01;32mroot@1fa96ba23b1b[00m:[01;34m/content/squad[00m# ls
 [0m[01;32mAnaconda3-5.3.0-Linux-x86_64.sh[0m   [01;34mdata[0m              models.py     setup.py
 args.py                           environment.yml   [01;34m__pycache__[0m   test.py
'Code Experiments.ipynb'           layers.py         README.md     train.py
 CS_72_Final_Project.ipynb         LICENSE           [01;34msave[0m          [...]
{'fraction_non_alphanumeric': 0.22914503288335064, 'fraction_numerical': 0.0680166147455867, 'mean_word_length': 3.571993670886076, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 33, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 26, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '621166', 'license': 'MIT', 'n_tokens_mistral': 3027, 'n_tokens_neox': 2745, 'n_words': 441, 'path': 'baseline-model/Baseline.ipynb', 'repo_name': 'Yakoob-Khan/squad', 'size': '5804'}