<jupyter_start><jupyter_text>Service Quality Monitoring in Confined Spaces Through Mining Twitter DataThe proposed method comprises of two main tasks.  Task 1: Aspect Extraction using fine-tuned BERT language modelIn this notebook, we use the BERT approach to transform tweets into a vector of words. Then, using a binary classifier, multi-label tweets can be classified into semantically-related groups, i.e., service quality aspects in our application.   Setup<jupyter_code>import warnings
warnings.filterwarnings("ignore")

from data_from_csv import *
from getting_data_ready import *
from PaddingInputExample import *
from InputExample import *
from InputFeatures import *
from BERT import *<jupyter_output><empty_output><jupyter_text>Training Phase  Loading Tweets<jupyter_code>csv_file = data_from_csv(os.path.join("Data",'SouthernCross.csv'))
data = csv_file.read_csv()
csv_file.report_on_classes()
data_ready = getting_data_ready(data, 0.1)
X_train, y_train, X_test, y_test = data_ready.splitting_data()<jupyter_output><empty_output><jupyter_text>Getting Ready the Train Data<jupyter_code>X_resampled, y_resampled = data_ready.resampling_data(X_train, y_train)
x_train_resampled = data_ready.resampled_to_table(X_resampled, y_resampled)
data_ready.report_on_resampled_classes()<jupyter_output>Safety:  4200
CleanlinessView:  4200
Information:  4200
Service:  6000
Comfort:  4200
PersonnelCard:  4800
Additional:  2400<jupyter_text>Getting Ready the Test Data<jupyter_code>test_data = data_ready.to_table(X_test, y_test)<jupyter_output><empty_output><jupyter_text>Bert Model<jupyter_code>bert_path = "uncased_L-12_H-768_A-12\\"
output_path = 'bert-output'
bert_class = BERT(bert_path, output_path)<jupyter_output>WARNING:tensorflow:From C:\Users\mmrahimi\AppData\Local\Continuum\anaconda3\lib\site-packages\bert\tokenization.py:125: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.<jupyter_text>Training Model<jupyter_code># google-pasta==0.1.6
# pip install gast==0.2.2
bert_class.train_bert(x_train_resampled)<jupyter_output>WARNING:tensorflow:From C:\Users\mmrahimi\OneDrive - The University of Melbourne\PhD\Tools\python_projects\BERT\BERT.py:446: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

WARNING:tensorflow:From C:\Users\mmrahimi\OneDrive - The University of Melbourne\PhD\Tools\python_projects\BERT\BERT.py:543: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

INFO:tensorflow:***** Running training *****
INFO:tensorflow:  Num examples = 16800
INFO:tensorflow:  Batch size = 32
INFO:tensorflow:  Num steps = 525
WARNING:tensorflow:From C:\Users\mmrahimi\OneDrive - The University of Melbourne\PhD\Tools\python_projects\BERT\BERT.py:480: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

INFO:tensorflow:Using config: {'_model_dir': 'bert-output', '_tf_random_seed': None, '_save_summary_steps': 500, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_sof[...]<jupyter_text>Testing model<jupyter_code>test_labels = bert_class.testing_model(test_data)<jupyter_output>Beginning Predictions!
Prediction took time  0:00:00.000995
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:num_labels:7;logits:Tensor("loss/BiasAdd:0", shape=(?, 7), dtype=float32);labels:Tensor("loss/Cast:0", shape=(?, 7), dtype=float32)
INFO:tensorflow:**** Trainable Variables ****
mode: infer probabilities: Tensor("loss/Sigmoid:0", shape=(?, 7), dtype=float32)
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from bert-output\model.ckpt-525
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
Classification report: 
               precision    recall  f1-score   support

           0       0.75      0.69      0.72        13
           1       0.67      0.46      0.55        13
           2       0.60      0.60      0.60        15
           3       0.87      0.89      0.88        74
           4       0.64      0.45      0.53        20
           5       0.50      0.14      0.22         7[...]<jupyter_text>Evaluating Phase<jupyter_code>def get_classification_report_as_df(report):
    cl_results = report.split()[4:39]
    cl_results_rest = report.split()[39:]
    df = pd.DataFrame(columns=['P','R','F'])
    for i in range(0,len(cl_results),5):
        df.loc[len(df)] = cl_results[i+1:i+4]

    for i in range(0,len(cl_results_rest),6):
        df.loc[len(df)] = cl_results_rest[i+2:i+5]

    return df


def mean_std_from_results(reports_dic):
    mean_result = pd.DataFrame(columns=['P','R','F'])
    std_result = pd.DataFrame(columns=['P','R','F'])
    for j in range(0,len(reports_dic[1])): # for each aspect
        df = pd.DataFrame(columns=['P','R','F'])
        for i in reports_dic: # for each fold
            df.loc[len(df)] = list(reports_dic[i].loc[j])
        df = df.apply(pd.to_numeric)
        mean_result.loc[len(mean_result)] = df.mean()
        std_result.loc[len(std_result)] = df.std()
    return mean_result, std_result<jupyter_output><empty_output><jupyter_text>Loading Tweets<jupyter_code>eval_csv_file = data_from_csv(os.path.join("Data",'Flinders.csv'))
eval_data = eval_csv_file.read_csv()
eval_csv_file.report_on_classes()<jupyter_output>Safety:  75
CleanlinessView:  21
Information:  194
Service:  948
Comfort:  117
PersonnelCard:  107
Additional:  16<jupyter_text>Evaluating Prediction<jupyter_code>eval_data = eval_data.reset_index(drop=True)
eval_labels = bert_class.testing_model(eval_data)<jupyter_output>Beginning Predictions!
Prediction took time  0:00:00.000997
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:num_labels:7;logits:Tensor("loss/BiasAdd:0", shape=(?, 7), dtype=float32);labels:Tensor("loss/Cast:0", shape=(?, 7), dtype=float32)
INFO:tensorflow:**** Trainable Variables ****
mode: infer probabilities: Tensor("loss/Sigmoid:0", shape=(?, 7), dtype=float32)
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from bert-output\model.ckpt-525
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
Classification report: 
               precision    recall  f1-score   support

           0       0.86      0.41      0.56        75
           1       0.43      0.29      0.34        21
           2       0.68      0.83      0.75       194
           3       0.92      0.92      0.92       948
           4       0.63      0.43      0.51       117
           5       0.74      0.50      0.59       107[...]
{'fraction_non_alphanumeric': 0.1135061802833886, 'fraction_numerical': 0.04687971058185107, 'mean_word_length': 4.163424124513619, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '456325', 'license': 'MIT', 'n_tokens_mistral': 2455, 'n_tokens_neox': 2262, 'n_words': 453, 'path': 'AspectExtraction.ipynb', 'repo_name': 'mmrahimi/SQ_monitoring_AE', 'size': '6634'}