<jupyter_start><jupyter_text>Command line spaCy training  Download datasets<jupyter_code>train_url = 'https://raw.githubusercontent.com/davidsbatista/NER-datasets/master/CONLL2003/train.txt'
valid_url = 'https://raw.githubusercontent.com/davidsbatista/NER-datasets/master/CONLL2003/valid.txt'
test_url = 'https://raw.githubusercontent.com/davidsbatista/NER-datasets/master/CONLL2003/test.txt'

train_file = 'train.txt'
valid_file = 'valid.txt'
test_file = 'test.txt'

!wget $train_url -O $train_file
!wget $valid_url -O $valid_file
!wget $test_url -O $test_file<jupyter_output>--2020-04-29 15:48:18--  https://raw.githubusercontent.com/davidsbatista/NER-datasets/master/CONLL2003/train.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 192.168.3.11, 172.16.17.32, 172.16.58.3, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|192.168.3.11|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3283418 (3.1M) [text/plain]
Saving to: ‘train.txt’


train.txt             0%[                    ]       0  --.-KB/s               
train.txt           100%[===================>]   3.13M  --.-KB/s    in 0.08s   

2020-04-29 15:48:18 (39.9 MB/s) - ‘train.txt’ saved [3283418/3283418]

--2020-04-29 15:48:19--  https://raw.githubusercontent.com/davidsbatista/NER-datasets/master/CONLL2003/valid.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 192.168.3.11, 172.16.17.32, 172.16.58.3, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|192.168.3.11|:443... conne[...]<jupyter_text>Convert datasets to spaCy json format<jupyter_code>!python -m spacy convert -c iob $train_file . -n 10
!python -m spacy convert -c ner $valid_file . -n 10
!python -m spacy convert -c ner $test_file . -n 10<jupyter_output>[38;5;4mℹ Auto-detected token-per-line NER format[0m
[38;5;4mℹ Grouping every 10 sentences into a document.[0m
[38;5;2m✔ Generated output file (347 documents): valid.json[0m<jupyter_text>Check json files<jupyter_code>import json

train_json = train_file[:-4] + '.json'
valid_json = valid_file[:-4] + '.json'
test_json = test_file[:-4] + '.json'

with open(train_json, 'r') as f:
  train_data = json.load(f)

train_data[0]
with open(valid_json, 'r') as f:
  valid_data = json.load(f)

valid_data[0]
with open(test_json, 'r') as f:
  test_data = json.load(f)

test_data[0]<jupyter_output><empty_output><jupyter_text>Train Model<jupyter_code>!mkdir model
!python -m spacy train en /model train.json valid.json -n 10 -p ner<jupyter_output>[38;5;3m⚠ Output directory is not empty[0m
This can lead to unintended side effects when saving the model. Please use an
empty directory or a different path instead. If the specified output path
doesn't exist, the directory will be created for you.
Training pipeline: ['ner']
Starting with blank model 'en'
Counting training words (limit=0)

Itn  NER Loss   NER P   NER R   NER F   Token %  CPU WPS
---  ---------  ------  ------  ------  -------  -------
  1  20829.916  80.372  80.007  80.189  100.000    26424
  2  10376.336  84.444  83.777  84.109  100.000    28081
  3   7287.740  86.182  85.544  85.861  100.000    27579
  4   5450.421  87.043  86.486  86.763  100.000    27748
  5   4229.981  87.534  87.210  87.371  100.000    27902
  6   3497.547  88.030  87.630  87.830  100.000    27971
  7   2908.579  88.192  87.984  88.088  100.000    27775
  8   2802.094  88.032  87.765  87.898  100.000    27694
  9   2415.713  88.369  87.967  88.167  100.000    27469
 10   2371.382  88.502  88.08[...]<jupyter_text>Evaluate model<jupyter_code>!python -m spacy evaluate /model/model-best test.json -R<jupyter_output>[1m
================================== Results ==================================[0m

Time      1.75 s
Words     46666 
Words/s   26714 
TOK       100.00
POS       0.00  
UAS       0.00  
LAS       0.00  
NER P     81.30 
NER R     81.44 
NER F     81.37 
Textcat   0.00
{'fraction_non_alphanumeric': 0.1581489956775998, 'fraction_numerical': 0.14950419527078565, 'mean_word_length': 3.0809128630705396, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 18, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 8, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '620748', 'license': 'MIT', 'n_tokens_mistral': 1884, 'n_tokens_neox': 1500, 'n_words': 352, 'path': 'playground/spaCy_training.ipynb', 'repo_name': 'artificiallama/ner', 'size': '3949'}