<jupyter_start><jupyter_text>CLASSIFY YELP REVIEWS (NLP)  PROBLEM STATEMENT - In this project, Natural Language Processing (NLP) strategies will be used to analyze Yelp reviews data- Number of 'stars' indicate the business rating given by a customer, ranging from 1 to 5- 'Cool', 'Useful' and 'Funny' indicate the number of cool votes given by other Yelp Users. Photo Credit: https://commons.wikimedia.org/wiki/File:Yelp_Logo.svg ![image.png](attachment:image.png)  LIBRARIES IMPORT<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>IMPORT DATASET<jupyter_code>yelp_df = pd.read_csv("yelp.csv")
yelp_df.head()
yelp_df.describe()
yelp_df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 10000 entries, 0 to 9999
Data columns (total 10 columns):
 #   Column       Non-Null Count  Dtype 
---  ------       --------------  ----- 
 0   business_id  10000 non-null  object
 1   date         10000 non-null  object
 2   review_id    10000 non-null  object
 3   stars        10000 non-null  int64 
 4   text         10000 non-null  object
 5   type         10000 non-null  object
 6   user_id      10000 non-null  object
 7   cool         10000 non-null  int64 
 8   useful       10000 non-null  int64 
 9   funny        10000 non-null  int64 
dtypes: int64(4), object(6)
memory usage: 781.4+ KB<jupyter_text>VISUALIZE DATASET<jupyter_code># Create a length column that contains the length of the review
yelp_df['length'] = yelp_df['text'].apply(len)
yelp_df.head()
yelp_df['length'].plot(bins=100, kind='hist');
yelp_df['length'].describe()
# Let's see the longest message 
yelp_df[yelp_df.length == 4997]['text'].iloc[0]
# Let's see the shortest message 
yelp_df[yelp_df['length'] == 1]['text'].iloc[0]
# Let's see the message with mean length 
yelp_df[yelp_df['length'] == 710]['text'].iloc[0]
sns.countplot(y='stars', data=yelp_df)
plt.show()
g = sns.FacetGrid(data=yelp_df, col='stars', col_wrap=3)
g.map(plt.hist, 'length', bins = 20, color = 'r')<jupyter_output><empty_output><jupyter_text>CREATE TESTING AND TRAINING DATASET/DATA CLEANING<jupyter_code># Let's define a pipeline to clean up all the messages 
# The pipeline performs the following: (1) remove punctuation, (2) remove stopwords
def message_cleaning(message):
    msg = "".join([char for char in message if char not in string.punctuation])
    return [word for word in msg.split() if word.lower() not in stopwords.words('english')]
yelp_df_1 = yelp_df[yelp_df['stars']==1]
yelp_df_5 = yelp_df[yelp_df['stars']==5]
yelp_df_1_5 = pd.concat([yelp_df_1 , yelp_df_5])
# Apply CountVectorizer
from sklearn.feature_extraction.text import CountVectorizer
vectorizer = CountVectorizer(analyzer=message_cleaning)
yelp_count_vectorizer = vectorizer.fit_transform(yelp_df_1_5['text'])
print(yelp_count_vectorizer.toarray())
yelp_count_vectorizer.shape<jupyter_output><empty_output><jupyter_text>TRAINING THE MODEL WITH ALL DATASET<jupyter_code>X = yelp_count_vectorizer
y = yelp_df_1_5['stars'].values

X.shape, y.shape
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
from sklearn.naive_bayes import MultinomialNB

NB_classifier = MultinomialNB()
NB_classifier.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>EVALUATING THE MODEL<jupyter_code>from sklearn.metrics import classification_report, confusion_matrix
y_pred_train = NB_classifier.predict(X_train)
y_pred_train
cm = confusion_matrix(y_train, y_pred_train)
sns.heatmap(cm, annot=True, fmt='g');
# preciting the test set
y_pred_test = NB_classifier.predict(X_test)
cm = confusion_matrix(y_test, y_pred_test)
sns.heatmap(cm, annot=True, fmt='g');
print(classification_report(y_test, y_pred_test))<jupyter_output>precision    recall  f1-score   support

           1       0.78      0.73      0.75       146
           5       0.94      0.96      0.95       672

    accuracy                           0.91       818
   macro avg       0.86      0.84      0.85       818
weighted avg       0.91      0.91      0.91       818
{'fraction_non_alphanumeric': 0.10777724435909745, 'fraction_numerical': 0.04320691310609698, 'mean_word_length': 4.150803461063041, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '478507', 'license': 'MIT', 'n_tokens_mistral': 1649, 'n_tokens_neox': 1465, 'n_words': 363, 'path': 'Beginner/CLASSIFY YELP REVIEWS (NLP).ipynb', 'repo_name': 'Jaygaur99/ML-Projects', 'size': '4166'}