<jupyter_start><jupyter_text>Sentimet Analysis Demo tutorial for how to use nlp_toolkit to train classification model and predict new samples. The task we choose is sentiment binary classification.The dataset is crawled from Kanzhun.com and Dajie.com, which is about company pros and cons. Available models:    1. Bi-LSTM Attention    2. Multi Head Self Attention    3. TextCNN    4. DPCNN<jupyter_code>from nlp_toolkit import Dataset, Classifier<jupyter_output>/opt/wangyilei/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
2018-12-18 23:35:27,866 - __init__.py[line:24] - INFO: Choose the most free GPU: 0, currently not support multi-gpus<jupyter_text>Data Processing  Load config dict<jupyter_code>import yaml
config = yaml.load(open('../config_classification.yaml'))
config['data']['basic_token'] = 'word'
config['embed']['pre'] = True
config['embed']['word']['path'] = '../data/embeddings/tencent_np_200d.txt'<jupyter_output><empty_output><jupyter_text>Load Data<jupyter_code>dataset = Dataset(fname='../data/company_pro_con_new.txt', task_type='classification', mode='train', config=config)
for x, y in zip(dataset.texts['token'][0:10], dataset.labels[0:10]):
    print(x, y)<jupyter_output>['进去', '前', '许诺', '的', '工资', '给', '的', '高', 8] ['pos']
['校园', '环境', '优美', '，', '美女', '很多', '，', '适合', '居住', '，', '食堂', '饭菜', '便宜', '，', '操场', '好', '，', '可以', '天天', '运动', 20] ['pos']
['老板', '人', '很好', '老', '员工', '会', '各种', '教', '你', '东西', '，', '而且', '不会', '有所', '保留', '薪水', '在', '大连', '还', '算', '可以', 21] ['pos']
['人员', '比较', '多', '，', '复杂', '办公室', '容易', '形成', '拉帮结派', '不利于', '企业', '发展', 12] ['neg']
['出差', '太多', '了', '。', '在', '现场', '开发', '很苦', '逼', '。', 10] ['neg']
['公司', '目前', '地理位置', '不', '太', '理想', '，', '离', '城市', '中心', '较', '远点', '。', 13] ['neg']
['公司', '的', '技术', '水平', '国内', '顶尖', '，', '十几', '年', '的', '资历', '，', '制作', '的', '作品', '几乎', '都', '是', '精品', '，', '参与', '过', '很多', '知名', '项目', '。', 26] ['pos']
['工作', '流程', '复杂', '个人', '上升', '空间', '有限', '新产品', '的', '创新', '能力', '有限', '组织', '架构', '稍', '显', '臃肿', 17] ['neg']
['无偿', '加班', '，', '加班', '多', '，', '没', '加班费', '，', '压力', '很大', 11] ['neg']
['环境', '比较', '轻松', '，', '跟', '项目', '走', '，', '能', '学', '不少', '专业', '知识', '，', '经验', '很', '重要', 17] [[...]<jupyter_text>if your want to see the vocab and label index mapping dict<jupyter_code># dataset.transformer._word_vocab._token2id
# dataset.transformer._label_vocab._token2id
transformer = dataset.transformer<jupyter_output><empty_output><jupyter_text>Classifier Training  Define classifier avialiable models: model_name_list = ['dpcnn', 'text_cnn', 'bi_lstm_att', 'multi_head_self_att']<jupyter_code>model_name='transformer'
# if you want to get attention weights during prediction, please set return_attention=True, only for bi_lstm_att model
# config['model'][model_name]['return_att'] = True
# text_cnn and dpcnn currently not support bucket sequence type
text_classifier = Classifier(model_name=model_name, dataset=dataset, seq_type='bucket')<jupyter_output><empty_output><jupyter_text>Train Model<jupyter_code>trained_model, history = text_classifier.train()<jupyter_output>2018-12-18 20:11:21,154 - trainer.py[line:123] - INFO: transformer model structure...
2018-12-18 20:11:21,201 - trainer.py[line:135] - INFO: train/valid set: 75708/18927
2018-12-18 20:11:21,202 - trainer.py[line:92] - INFO: use bucket sequence to speed up model training
2018-12-18 20:11:21,211 - sequence.py[line:355] - INFO: Training with 99 non-empty buckets<jupyter_text>plot acc and loss<jupyter_code>from nlp_toolkit import visualization as vs
vs.plot_loss_acc(history, 'classification')<jupyter_output><empty_output><jupyter_text>10-fold training<jupyter_code>config['train']['train_mode'] = 'fold'
text_classifier_new = Classifier(model_name=model_name, dataset=dataset, seq_type='bucket')
text_classifier_new.train()<jupyter_output>2018-12-18 20:59:08,543 - trainer.py[line:189] - INFO: 10-fold starts!<jupyter_text>Predict New Samples  Load data and transformer<jupyter_code>dataset = Dataset(mode='predict', fname='../data/meituan_consulting_reviews_hanlp.txt',
                  task_type='classification',
                  tran_fname='models/bi_lstm_att_201812171626/transformer.h5')<jupyter_output><empty_output><jupyter_text>Load model<jupyter_code>text_classifier = Classifier('bi_lstm_att', dataset)
text_classifier.load(weight_fname='models/bi_lstm_att_201812171626/model_weights_06_0.8413_0.9384.h5',
                     para_fname='models/bi_lstm_att_201812171626/model_parameters.json')<jupyter_output><empty_output><jupyter_text>predict samples<jupyter_code># if you want to get the label probability of each tokens, set return_prob as True
# currently only bi_lstm_att model supports return_attention, otherwise you will get errors
# if you don't want to use Dataset class to generate some other features or clean texts (not recommend)
# just make one fake dict with one key "token" and the value is the nested list of words
y_pred, attentions = text_classifier.predict(dataset.texts, return_prob=True, return_attention=True)<jupyter_output><empty_output><jupyter_text>attention visualization currently only support for bi_lstm_att model<jupyter_code>vs.att2html(dataset.texts['token'][-1], attentions[-1])<jupyter_output><empty_output><jupyter_text>公司 专业 技能 培训 机会 多 工作 ， 能 使 自己 快速 提升 ， 待遇 颇 丰 老板 人 很好 老 员工 会 各种 教 你 东西 ， 而且 不会 有所 保留 薪水 在 大连 还 算 可以  很好 ， 帮助 挺 大 的 or you can write all results to html file and open it in a browser<jupyter_code>vs.attention_visualization(dataset.texts['token'], attentions, output_fname='result.html')<jupyter_output><empty_output><jupyter_text>Evaluate model<jupyter_code>dataset = Dataset(mode='eval', fname='../data/company_pro_con_dev.txt',
                  task_type='classification',
                  tran_fname='models/bi_lstm_att_201812171626/transformer.h5')
text_classifier = Classifier('bi_lstm_att', dataset)
text_classifier.load(weight_fname='models/bi_lstm_att_201812171626/model_weights_06_0.8413_0.9384.h5',
                     para_fname='models/bi_lstm_att_201812171626/model_parameters.json')

# if you don't want to use Dataset class to generate some other features or clean texts (not recommend)
# just make one fake dict with one key "token" and the value is the nested list of words.
text_classifier.evaluate(dataset.texts, dataset.labels)<jupyter_output>2018-12-18 23:37:50,775 - data.py[line:118] - INFO: transformer loaded
2018-12-18 23:37:51,011 - data.py[line:154] - INFO: data loaded
{'fraction_non_alphanumeric': 0.1720605700712589, 'fraction_numerical': 0.047505938242280284, 'mean_word_length': 5.988589211618257, 'pattern_counts': {'":': 0, '<': 52, '<?xml version=': 0, '>': 52, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '582355', 'license': 'MIT', 'n_tokens_mistral': 2852, 'n_tokens_neox': 2726, 'n_words': 607, 'path': 'examples/sentiment.ipynb', 'repo_name': 'stevewyl/nlp_toolkit', 'size': '6736'}