<jupyter_start><jupyter_text>2019/09/28 由曾元顯建立，2020/08/24修改。需先下載 Google 釋出的 BERT 程式與中文模型，若不清楚，可參考：https://github.com/SamTseng/Chinese_Skewed_TxtClf/blob/master/BERT_txtclf_HowTo.docx 。 壹、設置並查詢 GPU在 Colab 畫面中，點擊 編輯 > 筆記本設定 或者 執行階段 > 變更執行階段類型 ，選擇 GPU 做為 硬體加速器<jupyter_code># 查詢 GPU
!nvidia-smi<jupyter_output>Mon Aug 24 09:57:15 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.57       Driver Version: 418.67       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |
| N/A   35C    P8    25W / 149W |      0MiB / 11441MiB |      0%      Default |
|                               |                      |                 ERR! |
+-------------------------------+----------------------+----------------------+
                                                                               
+-------[...]<jupyter_text>貳、設定 Colab 權限以連結本機上的雲端硬碟 運行下面的程式碼，會出現鏈接，點開鏈接，選擇自己的 Google 帳號，將得到的代碼（類似：「4/rgFOTdBFqoSFMoinr_9uzzTLziWUyGjSQgNLt1EV4gbrM6gBYp-Vnxk」），拷貝後輸入鏈接下面的框中即可。<jupyter_code>from google.colab import drive
drive.mount('/content/gdrive')<jupyter_output><empty_output><jupyter_text>設置工作路徑：<jupyter_code>import os, time
os.chdir( "/content/gdrive/My Drive/BERT/lis" ) # change directory to your GoogleDrive<jupyter_output><empty_output><jupyter_text>將筆記本的工作路徑設置到BERT代碼的文件夾下。可以用ls命令查看是否成功：<jupyter_code>!pwd
!ls<jupyter_output>/content/gdrive/My Drive/BERT/lis
chinese_L-12_H-768_A-12  Model_LIS			 run_classifier_lis.py
data			 optimization.py		 tokenization.py
eval_clf_lis.py		 Output_LIS			 train_test_split.py
extract_features.py	 __pycache__
modeling.py		 run_classifier_lis_colab.ipynb<jupyter_text>參、安裝舊版的 tesorflow 以便執行 Google 釋出的程式由於現在連上 Colab，都使用新版 tensorflow，要先解除。再安裝舊版的 tensorflow，以搭配 Google 釋出的程式：https://github.com/google-research/bert<jupyter_code>!pip uninstall tensorflow
# 會被問：Proceed (y/n)? 記得要按「y]

!pip install tensorflow==1.15.0<jupyter_output>Collecting tensorflow==1.15.0
  Using cached https://files.pythonhosted.org/packages/3f/98/5a99af92fb911d7a88a0005ad55005f35b4c1ba8d75fba02df726cd936e6/tensorflow-1.15.0-cp36-cp36m-manylinux2010_x86_64.whl
Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.15.0)
Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.15.1)
Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.31.0)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.34.2)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.15.0)
Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.12.1)
Requirement already satisfied: keras-[...]<jupyter_text>肆、訓練、預測、評估 1. 訓練 lis注意1：當前的目錄下，至少要有這幾個 Google 釋出的檔案：extract_features.pymodeling.pyoptimization.pytokenization.pychinese_L-12_H-768_A-12：最好改成多國語言版：https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip注意2：run_classifier_lis.py 裡面，需要讀取 LIS_train.txt 與 LIS_test.txt 兩個檔案，請確保其檔名正確，路徑也正確。訓練時間大約花費 1500 秒（K80 GPU）。若訓練時間很多，例如不到60秒，可能已經訓練過，或是訓練資料有誤，請刪除 Model_LIS 目錄後，等待同步完成，再重新訓練。<jupyter_code>time_Start = time.time()
!python run_classifier_lis.py \
  --task_name=lis --do_train=true --do_eval=false \
  --data_dir=data \
  --vocab_file=multi_cased_L-12_H-768_A-12/vocab.txt \
  --bert_config_file=multi_cased_L-12_H-768_A-12/bert_config.json \
  --init_checkpoint=multi_cased_L-12_H-768_A-12/bert_model.ckpt \
  --max_seq_length=256 \
  --train_batch_size=8 \
  --learning_rate=2e-5 \
  --num_train_epochs=10.0 \
  --output_dir=Model_LIS
print("It takes %4.2f seconds to train lis."%(time.time()-time_Start))<jupyter_output><empty_output><jupyter_text>2、預測 lis預測時間約需 42 秒。<jupyter_code>time_Start = time.time()
!python run_classifier_lis.py \
  --task_name=lis --do_predict=true \
  --data_dir=data \
  --vocab_file=multi_cased_L-12_H-768_A-12/vocab.txt \
  --bert_config_file=multi_cased_L-12_H-768_A-12/bert_config.json \
  --init_checkpoint=Model_LIS \
  --max_seq_length=256 \
  --output_dir=Output_LIS
print("It takes %4.2f seconds to predict CnonC."%(time.time()-time_Start))<jupyter_output><empty_output><jupyter_text>3.評估 lis 成效<jupyter_code>!python eval_clf_lis.py lis data/LIS_test.txt Output_LIS/test_results.tsv<jupyter_output>Print out the F1-score of the result of BERT classifier for a data set.
  Usage:   python eval_clf.py Data_Name Test_File Predicted_File
  Example: python eval_clf.py CnonC CnonC_test.txt Output_CnonC/test_results.tsv

sys.argv: ['eval_clf_lis.py', 'lis', 'data/LIS_test.txt', 'Output_LIS/test_results.tsv']
E.資訊服務與使用者研究      147
H.數位典藏與數位學習研究     110
I.資訊與社會            96
G.資訊系統與檢索          93
F.圖書館與資訊服務機構管理     92
J.資訊計量學            58
C.館藏發展             44
B.圖書資訊學教育          35
D.資訊與知識組織          27
A.圖書資訊學理論與發展        5
Name: 0, dtype: int64
From 'data/LIS_test.txt' file: df0
Label_List='['E.資訊服務與使用者研究', 'H.數位典藏與數位學習研究', 'I.資訊與社會', 'G.資訊系統與檢索', 'F.圖書館與資訊服務機構管理', 'J.資訊計量學', 'C.館藏發展', 'B.圖書資訊學教育', 'D.資訊與知識組織', 'A.圖書資訊學理論與發展']'
Number of Labels: 10
From copy of get_labels() in BERT's run_classifier.py
Label_List='['A.圖書資訊學理論與發展', 'B.圖書資訊學教育', 'C.館藏發展', 'D.資訊與知識組織', 'E.資訊服務與使用者研究', 'F.圖書館與資訊服務機構管理', 'G.資訊系統與檢索', 'H.數位典藏與數位學習研究', 'I.資訊與社會', 'J.資訊計量學']'
Number of Labels:10
Num of Classes [...]
{'fraction_non_alphanumeric': 0.17745722911774572, 'fraction_numerical': 0.05904059040590406, 'mean_word_length': 3.14673157162726, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 36, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '525346', 'license': 'MIT', 'n_tokens_mistral': 3027, 'n_tokens_neox': 3042, 'n_words': 343, 'path': 'run_classifier_lis_colab.ipynb', 'repo_name': 'SamTseng/LIS_Yearbook', 'size': '5962'}