<jupyter_start><jupyter_text>Sentiment Analysis with NLP for Hotel Accommodation Reviews- <NAME>- 2017/11/09  IntroductionRecently I got hotel accommodation reviews data to practice **Sentiment Analysis with Natural Language Processing (NLP)**, which I previously just knew the basics and would like to gain hands-on experience for this **Natural Language Understanding** task. This notebook is to summarize my results.  Sentiment Analysis with NLPWe will perform Sentiment Analysis with NLP by applying the **Occam's Razor Principle.**- Collect datasets- Exploratory data analysis (EDA) with datasets  * Check missing / abnormal data  * Group-by aggregate score distributions- Pre-process datasets  * Remove missing / abnormal data  * Join score & review datasets  * Concat review_title and review_comment to review_title_comments  * Lower review_title_comments  * Tokenize and remove stopwords and punctuations  * Get bag of words- Sentiment analysis  * Randomly permutate data  * Label review  * Splite training and test sets  * Machine learning for classification by Naive Bayes Classifier  * Model evaluation by precision and recall- Measure real-world performance  * Predict label based on bag of words  * Compare two labels's score distributions  Python scriptsFirst import Python libraries.<jupyter_code>from __future__ import division
from __future__ import print_function
import numpy as np
import scipy as sp
import pandas as pd

import nltk
# When performing experiment, remove comment out for nltk.download().
# nltk.download()

import time
import warnings
warnings.filterwarnings("ignore")<jupyter_output><empty_output><jupyter_text>The following are the scripts for Sentiment Analysis with NLP.<jupyter_code>score_file = 'reviews_score.csv'
review_file = 'reviews.csv'
def read_score_review(score_file, review_file):
    """Read score and review data."""
    score_df = pd.read_csv(score_file)
    review_df = pd.read_csv(review_file)
    return score_df, review_df
def groupby_agg_data(df, gkey='gkey', rid='rid'):
    """Group-by aggregate data."""
    agg_df = (df.groupby(gkey)[rid]
                .count()
                .reset_index())
    nan_count = df[gkey].isnull().sum()
    nan_df = pd.DataFrame({gkey: [np.nan], rid: [nan_count]})
    agg_df = agg_df.append(nan_df)[[gkey, rid]]
    agg_df['percent'] = agg_df[rid] / agg_df[rid].sum()
    return agg_df
def count_missing_data(df, cols='cols'):
    """Count missing records w.r.t. columns."""
    print('Missing rows:')
    for col in cols:
        nan_rows = df[col].isnull().sum()
        print('For {0}: {1}'.format(col, nan_rows))
def slice_abnormal_id(df, rid='hotel_review_id'):
    """View abnormal records with column"""
    abnorm_bool_arr = (df[rid] == 0)
    abnorm_count = abnorm_bool_arr.sum()
    print('abnorm_count: {}'.format(abnorm_count))
    abnorm_df = df[abnorm_bool_arr]
    return abnorm_df
def remove_missing_abnormal_data(score_raw_df, review_raw_df, 
                                 rid='hotel_review_id', 
                                 score_col='rating_overall'):
    """Remove missing / abnormal data."""
    filter_score_bool_arr = (score_raw_df[rid].notnull() & 
                             score_raw_df[score_col].notnull())
    score_df = score_raw_df[filter_score_bool_arr]
    
    filter_review_bool_arr = review_raw_df[rid].notnull()
    review_df = review_raw_df[filter_review_bool_arr]
    
    return score_df, review_df
def join_score_review(score_df, review_df, on='hotel_review_id', how='left'):
    """Join score and review datasets."""
    score_review_df = pd.merge(score_df, review_df, on=on, how=how)
    score_review_count = score_review_df.shape[0]
    print('score_review_count: {}'.format(score_review_count))
    return score_review_df
def concat_review_title_comments(score_review_df, 
                                 concat_cols=['review_title', 'review_comments'],
                                 concat_2col='review_title_comments'):
    """Concat review title and review comments."""
    concat_text_col = ''
    for concat_col in concat_cols:
        concat_text_col += score_review_df[concat_col]
        if concat_col != concat_cols[len(concat_cols) - 1]:
            concat_text_col += '. '
    score_review_df[concat_2col] = concat_text_col
    return score_review_df
def lower_review_title_comments(score_review_df, 
                                lower_col='review_title_comments'):
    """Lower sentences."""
    score_review_df[lower_col] = score_review_df[lower_col].str.lower()
    return score_review_df
def _tokenize_sen(sen):
    """Tokenize one sentence."""
    from nltk.tokenize import word_tokenize
    sen_token = word_tokenize(str(sen))
    return sen_token
def _remove_nonstop_words_puncs(sen):
    """Remove nonstop words and meaningless punctuations in one sentence."""
    from nltk.corpus import stopwords
    sen_clean = [
        word for word in sen 
        if word not in stopwords.words('english') and 
           word not in [',', '.', '(', ')', '&']]
    return sen_clean
def tokenize_clean_sentence(sen):
    """Tokenize and clean one sentence."""
    sen_token = _tokenize_sen(sen)
    sen_token_clean = _remove_nonstop_words_puncs(sen_token)
    return sen_token_clean
# def preprocess_sentence(df, sen_cols=['review_title', 'review_comments']):  
#     """Preprocess sentences (deprecated due to slow performance)."""
#     for sen_col in sen_cols:
#         print('Start tokenizing "{}"'.format(sen_col))
#         sen_token_col = '{}_token'.format(sen_col)
#         df[sen_token_col] = df[sen_col].apply(tokenize_clean_sentence)
#         print('Finish tokenizing "{}"'.format(sen_col))
#     return df

def preprocess_sentence_par(df, sen_col='review_title_comments',
                            sen_token_col='review_title_comments_token', num_proc=32):
    """Preporecess sentences in parallel.
    
    Note: We apply multiprocessing with 32 cores; adjust `num_proc` by your computing environment.
    """
    import multiprocessing as mp
    pool = mp.Pool(num_proc)
    df[sen_token_col] = pool.map_async(tokenize_clean_sentence , df[sen_col]).get()
    return df
def get_bag_of_words(w_ls):
    """Get bag of words in word list."""
    w_bow = dict([(w, True) for w in w_ls])
    return w_bow
def get_bag_of_words_par(df, sen_token_col='review_title_comments_token',
                         bow_col='review_title_comments_bow', num_proc=32):
    """Get bag of words in parallel for sentences."""
    import multiprocessing as mp
    pool = mp.Pool(num_proc)
    df[bow_col] = pool.map_async(get_bag_of_words , df[sen_token_col]).get()
    return df
def label_review(df, scores_ls=None, label='negative',
                 score_col='rating_overall',
                 review_col='review_title_comments_bow'):
    """Label review by positive or negative."""
    df_label = df[df[score_col].isin(scores_ls)]
    label_review_ls = (df_label[review_col]
                       .apply(lambda bow: (bow, label))
                       .tolist())
    return label_review_ls
def permutate(data_ls):
    """Randomly permutate data."""
    np.random.shuffle(data_ls)
def create_train_test_sets(pos_review_ls, neg_review_ls, train_percent=0.75):
    """Create the training and test sets."""
    neg_num = np.int(np.ceil(len(neg_review_ls) * train_percent))
    pos_num = np.int(np.ceil(len(pos_review_ls) * train_percent))
    
    train_set = neg_review_ls[:neg_num] + pos_review_ls[:pos_num]
    permutate(train_set)
    
    test_set =  neg_review_ls[neg_num:] + pos_review_ls[pos_num:]
    permutate(test_set)
    
    return train_set, test_set
def train_naive_bayes(train_set):
    from nltk.classify import NaiveBayesClassifier
    nb_clf = NaiveBayesClassifier.train(train_set)
    return nb_clf
def eval_naive_bayes(test_set, nb_clf):
    import collections
    from nltk.metrics.scores import precision
    from nltk.metrics.scores import recall

    ref_sets = {'positive': set(), 
                'negative': set()}
    pred_sets = {'positive': set(), 
                 'negative': set()}
    
    for i, (bow, label) in enumerate(test_set):
        ref_sets[label].add(i)
        pred_label = nb_clf.classify(bow)
        pred_sets[pred_label].add(i)
        
    print('Positive precision:', precision(ref_sets['positive'], pred_sets['positive']))
    print('Positive recall:', recall(ref_sets['positive'], pred_sets['positive']))
    print('Negative precision:', precision(ref_sets['negative'], pred_sets['negative']))
    print('Negative recall:', recall(ref_sets['negative'], pred_sets['negative']))
def pred_labels(df, clf, 
                bow_col='review_title_comments_bow',
                pred_col='pred_label',
                sel_cols=['rating_overall', 
                          'review_title_comments_bow', 
                          'pred_label']):
    """Predict labels for bag of words."""
    df[pred_col] = df[bow_col].apply(clf.classify)
    df_pred = df[sel_cols]
    return df_pred
def get_boxplot_data(pred_label_df, 
                     pred_col='pred_label', score_col='rating_overall'):
    pos_data = pred_label_df[pred_label_df[pred_col] == 'positive'][score_col].values
    neg_data = pred_label_df[pred_label_df[pred_col] == 'negative'][score_col].values
    box_data = [pos_data, neg_data]
    return box_data
def plot_box(d_ls, title='Box Plot', xlab='xlab', ylab='ylab', 
             xticks=None, xlim=None, ylim=None, figsize=(15, 10)):
    import matplotlib.pyplot as plt
    import seaborn as sns
    import matplotlib
    matplotlib.style.use('ggplot')
    %matplotlib inline
    plt.figure()
    fig, ax = plt.subplots(figsize=figsize)
    plt.boxplot(d_ls)
    plt.title(title)
    plt.xlabel(xlab)
    plt.ylabel(ylab)
    if xticks:
        ax.set_xticklabels(xticks)
    if xlim:
        plt.xlim(xlim)
    if ylim:
        plt.ylim(ylim)
    # plt.axis('auto')    
    plt.show()<jupyter_output><empty_output><jupyter_text>Collect DataWe first read score and review raw datasets.- Score dataset: two columns  * `hotel_review_id`: hotel review sequence ID  * `rating_overall`: overal accommodation rating- Review dataset: three columns  * `hotel_review_id`: hotel review sequence ID  * `review_title`: review title  * `review_comments`: detailed review comments<jupyter_code>score_raw_df, review_raw_df = read_score_review(score_file, review_file)

print(len(score_raw_df))
print(len(review_raw_df))
score_raw_df.head(5)
review_raw_df.head(5)<jupyter_output><empty_output><jupyter_text>EDA with Datasets   Check missing / abnormal data<jupyter_code>count_missing_data(score_raw_df, 
                   cols=['hotel_review_id', 'rating_overall'])
score_raw_df[score_raw_df.rating_overall.isnull()]
count_missing_data(review_raw_df, 
                   cols=['hotel_review_id', 'review_title', 'review_comments'])
abnorm_df = slice_abnormal_id(score_raw_df, rid='hotel_review_id')

abnorm_df
abnorm_df = slice_abnormal_id(review_raw_df, rid='hotel_review_id')

abnorm_df<jupyter_output>abnorm_count: 24<jupyter_text>Group-by aggregate score distributionsFrom the following results we can observe that - the `rating_overall` scores are **imbalanced.** Specifically, only about $1\%$ records have low scores $\le 5$, thus about $99\%$ records have scores $\ge 6$.- some records have **missing score**.<jupyter_code>score_raw_df.rating_overall.unique()
score_agg_df = groupby_agg_data(
    score_raw_df, gkey='rating_overall', rid='hotel_review_id')

score_agg_df<jupyter_output><empty_output><jupyter_text>Pre-process Datasets  Remove missing / abnormal dataSince there are few records (only 27) having missing `hotel_review_id` and `rating_overall` score, we just ignore them.<jupyter_code>score_df, review_df = remove_missing_abnormal_data(
    score_raw_df, review_raw_df, 
    rid='hotel_review_id', 
    score_col='rating_overall')
score_df.head(5)
review_df.head(5)<jupyter_output><empty_output><jupyter_text>Join score & review datasetsTo leverage fast vectorized operation with Pandas DataFrame, we joint score and review datasets.<jupyter_code>score_review_df_ = join_score_review(score_df, review_df)

score_review_df_.head(5)<jupyter_output>score_review_count: 274312<jupyter_text>The following are the procedure for processing natural language texts. Concat review_title and review_commentsUsing the Occam's Razor Principle, since `review_title` and `review_comments` both are natural languages, we can simply concat them into one sentence for further natural language processing.<jupyter_code>score_review_df = concat_review_title_comments(
    score_review_df_, 
    concat_cols=['review_title', 'review_comments'],
    concat_2col='review_title_comments')
score_review_df.head(5)<jupyter_output><empty_output><jupyter_text>Lower review_title_comments<jupyter_code>score_review_df = lower_review_title_comments(
    score_review_df, 
    lower_col='review_title_comments')
score_review_df.head(5)<jupyter_output><empty_output><jupyter_text>Tokenize and remove stopwords**Tokenizing** is an important technique by which we would like to split the sentence into vector of invidual words. Nevertheless, there are many **stopwords** that are useless in natural language text, for example: he, is, at, which, and on. Thus we would like to remove them from the vector of tokenized words.Note that since the tokenizing and removing stopwords tasks are time-consuming, we apply Python build-in package `multiprocessing` for parallel computing to improve the performance.<jupyter_code>start_token_time = time.time()

score_review_token_df = preprocess_sentence_par(
    score_review_df, 
    sen_col='review_title_comments',
    sen_token_col='review_title_comments_token', num_proc=32)

end_token_time = time.time()
print('Time for tokenizing: {}'.format(end_token_time - start_token_time))
score_review_token_df.head(5)
score_review_token_df.review_title_comments_token[1]<jupyter_output><empty_output><jupyter_text>Get bag of wordsThe tokenized words may contain duplicated words, and for simplicity, we would like to apply the **Bag of Words,** which just represents the sentence as a bag (multiset) of its words, ignoring grammar and even word order. Here, following the Occam's Razor Principle again, we do not keep word frequencies, thus we use binary (presence/absence or True/False) weights.<jupyter_code>start_bow_time = time.time()

score_review_bow_df = get_bag_of_words_par(
    score_review_token_df, 
    sen_token_col='review_title_comments_token',
    bow_col='review_title_comments_bow', num_proc=32)

end_bow_time= time.time()
print('Time for bag of words: {}'.format(end_bow_time - start_bow_time))
score_review_bow_df.review_title_comments_bow[:5]<jupyter_output><empty_output><jupyter_text>Sentiment Analysis  Label dataSince we would like to **polarize data** with consideration for the **imbalanced data problem** as mentioned before, we decide to label- ratings 2, 3 and 4 by "negative",- ratings 9 and 10 by "positive".<jupyter_code>neg_review_ls = label_review(
    score_review_bow_df,
    scores_ls=[2, 3, 4], label='negative',
    score_col='rating_overall',
    review_col='review_title_comments_bow')
pos_review_ls = label_review(
    score_review_bow_df,
    scores_ls=[9, 10], label='positive',
    score_col='rating_overall',
    review_col='review_title_comments_bow')
neg_review_ls[1]
pos_review_ls[1]<jupyter_output><empty_output><jupyter_text>Splite training and test setsWe split the training and test sets by the rule of $75\%$ and $25\%$.<jupyter_code>train_set, test_set = create_train_test_sets(
    pos_review_ls, neg_review_ls, train_percent=0.75)
train_set[10]<jupyter_output><empty_output><jupyter_text>Naive Bayes ClassificationWe first apply **Naive Bayes Classifier** to learn positive or negative sentiment.<jupyter_code>nb_clf = train_naive_bayes(train_set)<jupyter_output><empty_output><jupyter_text>Model evaluationWe evaluate our model by **positive / negative precision and recall.** From the results we can observe that our model performs fairly good.<jupyter_code>eval_naive_bayes(test_set, nb_clf)<jupyter_output>Positive precision: 0.976486961617
Positive recall: 0.917986503236
Negative precision: 0.62166454892
Negative recall: 0.859086918349<jupyter_text>Measure Real-World Performance Predict label based on bag of words<jupyter_code>start_pred_time = time.time()

pred_label_df = pred_labels(
    score_review_bow_df, nb_clf, 
    bow_col='review_title_comments_bow',
    pred_col='pred_label')

end_pred_time = time.time()
print('Time for prediction: {}'.format(end_pred_time - start_pred_time))
pred_label_df.head(5)<jupyter_output><empty_output><jupyter_text>Compare two labels's score distributionsFrom the following boxplot, we can observe that our model performs reasonably well in the real world, even by our suprisingly simple machine learning modeling. We can further apply divergence measures, such as Kullback-Leibler divergence, to quantify the rating_overall distribution distance between two label groups, if needed.<jupyter_code>box_data = get_boxplot_data(
    pred_label_df, 
    pred_col='pred_label', score_col='rating_overall')
plot_box(box_data, title='Box Plot for rating_overall by Sentiment Classes', 
         xlab='class', ylab='rating_overall', 
         xticks=['positive', 'negative'], figsize=(12, 7))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09892324523521621, 'fraction_numerical': 0.007830943743882076, 'mean_word_length': 4.6007739438890685, 'pattern_counts': {'":': 0, '<': 67, '<?xml version=': 0, '>': 67, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '145049', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 5693, 'n_tokens_neox': 5459, 'n_words': 1312, 'path': 'notebook/sentiment_nlp.ipynb', 'repo_name': 'bowen0701/data-science', 'size': '17369'}