<jupyter_start><jupyter_text>Predicting Zomato Restaurant Rate  1. Business Problem  1.1 Problem Description - Does demography of area matters?- Does location of particular type of restraurant depends on people living in that area>- Does theme of restraurant matters?- Is food chain category restraurant likely to have more customers than its counter part?- Are any neighbourhood on similar based on the type of food?- Is particular neighbours is famous for itw own kind of food?- If two neighbours are similar does that mean these are related or particular group of people live in neighbourhood or these are places to eat.- What kind of food is famous in locality.- Do entire locality loves veg food, if yes then locality populated by particular set of people eg Jain, Gujarati,Marwadi who are basically veg.  1.2. Problem StatementThe dataset also contains reviews for each of the restaurant which will help in finding overall rating for the place. So we will try to predict rating for particular restaurant.  1.3 Business Objectives  1.4 Machine Learning  Formulation  1.5 Performance Metrics  2. Data  2.1 Data DownloadZomato API was used to download data. Data was fetched from Zomato API by making zomatoWrapper package. Description on zomatoWrapper package can be found here:https://github.com/ashish-gh/zomatoWrapperZomato API provides data in json which was later stored in .csv file. Particularly two API endpoints were used to fetch data. Finally the data were merged and final csv file was produced. Data storage manipulation process can be found here:https://github.com/ashish-gh/Exploratory_Data_Analysis_Zomato_Restaurant/blob/master/DataStorage_and_Pre-processing.ipynb  2.2 Data Understanding<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import ast

from wordcloud import WordCloud, STOPWORDS
from sklearn.preprocessing import OneHotEncoder

from joblib import dump,load
#%matplotlib notebook
%matplotlib inline

import warnings
warnings.filterwarnings('ignore')

from sklearn.linear_model import LinearRegression
from sklearn import linear_model
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import OneHotEncoder
from sklearn.model_selection import train_test_split
from sklearn.metrics import  make_scorer
from sklearn.model_selection import GridSearchCV
from sklearn import metrics

import random
from xgboost import XGBClassifier
from sklearn.ensemble import RandomForestRegressor
data = pd.read_csv('zomato_final.csv')
data.shape
data.head()
data.info()
# country_code,address locality, locaity_verbose, switch_to_orvder_menu, switch_to_orvder_menu, rating_color,
data.votes.describe()<jupyter_output><empty_output><jupyter_text>Column description<jupyter_code>data.columns<jupyter_output><empty_output><jupyter_text>- restaurant_id: contains unique id for each restaurant- restaurant_name: contains the name of the restaurant- country_code: containts the country code in which the restaurant is situated- city: - address: contains the address of the restaurant- locality:- locality_verbose:- longitude: - latitude: - cuisines:- average_cost_for_two:- has_table_booking:- has_online_delivery:- is delivering_now:- switch_to_order_now:- price_range - aggregated_rating  2.3 Data preprocess<jupyter_code># explore columns related to the addrress
data.loc[:,['address','city','locality', 'locality_verbose']].sample(8,random_state=1)<jupyter_output><empty_output><jupyter_text>Here, we can see that among 4 columns 2 represent same information. So just dropping column which are not important.- we are going to keep the `city` column and drop the address, locality, locality_verbose- columns country_code, switch_to_order_menu, rating_color,<jupyter_code># drop unnecessary columns
column_to_drop = ['restaurant_id', 'address','locality', 'locality_verbose' ,'country_code','switch_to_orvder_menu', 'rating_color']
data.drop(columns=column_to_drop, axis=1,inplace=True)
data.columns<jupyter_output><empty_output><jupyter_text>2.3.1 Remove Duplicates**Q.1) Is there duplicate values present in dataset? If yes then many of them are duplicate?**<jupyter_code># check for duplicate values
print("No of Duplicates in dataset:  ",data.duplicated().sum())<jupyter_output>No of Duplicates in dataset:   0<jupyter_text>2.3.1 Removing Null values**Q.2) Is there NULL values present in dataset? If yes then many they are (in %)?**<jupyter_code># check for null values
((data.isna().sum()/data.shape[0])*100).round(2)
type(data.review)
# for restaurant name

name_all = ""
for name in data['review']:
    if isinstance(name,str) and len(name)>0:
        name = name.lower()
        name_all += str(name)+" "

def has_special_char(text: str) -> bool:
    return any(c for c in text if not c.isalnum() and not c.isspace())

special_character=""
for text in name_all:
        if has_special_char(text):
            special_character += str(text)        

# print(list(set(special_character)))
special_character = list(set(special_character))
len(special_character)

characters = [''.join(character for character in special_character)]
characters
import re
import nltk
from nltk.corpus import stopwords
nltk.download('stopwords')

def clean_text(text):
    text = text.lower()    
    
    text = re.sub(r'[_"\-;%()|+&=*%.,!?:#$@\[\]/]', '', text)
    
    text = re.sub(r'>\u200b“–\\`🍅‐♥•°„', '', text)
    
    text = re.sub(r'\'', '', text)
    
    text = re.sub(r'  ', ' ', text)
    
    text = re.sub(r'\d+', '', text)    
    
    text = text.split()
    
    text =  [word for word in text if not word in set(stopwords.words("english"))]
    
#     stops = set(stopwords.words("english"))
#     stops = set(stopwords.words("english"))
#     text = [w for w in text if not w in stops]
#     text = " ".join(text)

    text = ' '.join(text)


    return text

data.review = data.review.apply(lambda x: clean_text(x))
data.review
# def no_special_character(text: str) -> bool:
#     return any(c for c in text if c.isalnum() and c.isspace())
# # data['process_text'] = data.review.apply(lambda x: no_special_character(x))
# process_text=""
# for text in data.review:
#         if has_special_char(text):
#             process_text += str(text)        

# process_text
data.cuisines
data.cuisines = data.cuisines.apply(lambda x: clean_text(x))
data.cuisines<jupyter_output><empty_output><jupyter_text>2.4 Data Visualizations<jupyter_code># sns.distplot(data.aggregate_rating, kde=False, bins=20)

data.aggregate_rating.hist(color='blue')
plt.axvline(x= data.aggregate_rating.mean(),ls='--',color='red')
plt.title('Average Rating of Restaurants',weight='bold')
plt.xlabel('Rating')
plt.ylabel('No of Restaurants')
print("Mean is : ",data.aggregate_rating.mean())
# data.rating_text.hist(color='blue')
sns.countplot(x=data.rating_text, data=data)
plt.title('Reviews Restaurants',weight='bold')
plt.xlabel('Rating')
plt.ylabel('No of revies')<jupyter_output><empty_output><jupyter_text>Observation **Q.5) Which are the top 20 restaurant**<jupyter_code>plt.figure(figsize=(12,6))
ax =data.restaurant_name.value_counts()[:20].plot(kind='bar')
ax.legend(['* Restaurants'])
plt.xlabel('No of Restaurants')
plt.ylabel('Count of Restaurant')
plt.title("Name Vs Number of Restaurants", fontsize=20, weight='bold')<jupyter_output><empty_output><jupyter_text>3. Model<jupyter_code>data.info()
# drop unnecessary columns
column_to_drop = ['longitude','latitude']
data.drop(columns=column_to_drop, axis=1,inplace=True)<jupyter_output><empty_output><jupyter_text>3.1 Converting to dummy variables<jupyter_code># pd.get_dummies ==> Convert categorical variable into dummy/indicator variables.(OneHot encoding)

data['has_table_booking']= pd.get_dummies(data.has_table_booking, drop_first=True)
data['has_table_booking']= pd.get_dummies(data.has_table_booking, drop_first=True)
data['is_delivering_now']= pd.get_dummies(data.is_delivering_now, drop_first=True)

data 
#  cuisines converted to categorical features

data['cuisines'] = data['cuisines'].astype(str).apply(lambda x: ' '.join(sorted(x.split())))
data['cuisines'].value_counts().head()
data = data.rename(columns={
    'average_cost_for_two':'average_cost',
    'aggregate_rating':'rate'
    })<jupyter_output><empty_output><jupyter_text>3.2 Splitting data for Model Building<jupyter_code># build x & y dataset
x = data.drop(['restaurant_name','rate'],axis = 1)
y = data['rate']
x.shape, y.shape
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(x,y,test_size = 0.3,random_state = 33)
X_train.shape, X_test.shape, y_train.shape, y_test.shape
# One hot encoder
enc = OneHotEncoder( handle_unknown='ignore')
## ALWAYS AVOID DATA LEAKAGE

# this method is for training data set
def one_hot_fit_transform(df,name):
    output_data = df[name].values.reshape(-1, 1)
    return enc.fit_transform(output_data).toarray()

# this method is for test data set
def one_hot_transform(df,name):
    output_data1 = df[name].values.reshape(-1, 1)
    return enc.transform(output_data1).toarray()
# one hot encoding apply to 'rest_type' features on train/test dataset
tr_dummy_rating_text = one_hot_fit_transform(X_train,'rating_text' )
te_dummy_rating_text = one_hot_transform(X_test,'rating_text' )

# one hot encoding apply to 'location' features on train/test dataset
tr_dummy_city = one_hot_fit_transform(X_train,'city' )
te_dummy_city= one_hot_transform(X_test,'city')

# one hot encoding apply to 'cuisines' features on train/test dataset
tr_dummy_cuisines = one_hot_fit_transform(X_train,'cuisines' )
te_dummy_cuisines = one_hot_transform(X_test,'cuisines')
tr_dummy_rating_text.shape, te_dummy_rating_text.shape
tr_dummy_cuisines.shape, te_dummy_cuisines.shape
tr_dummy_city.shape, te_dummy_city.shape
## combine all 'one-hot' encoded features as Tr.
tr =pd.DataFrame(pd.np.column_stack([ tr_dummy_rating_text,tr_dummy_city, tr_dummy_cuisines]))

## CONCAT both dataframe ### ie Tr and X_train(original dataframe)
## https://stackoverflow.com/questions/45963799/pandas-concat-resulting-in-nan-rows

l1=X_train.values.tolist()
l2=tr.values.tolist()

for i in range(len(l1)):
    l1[i].extend(l2[i])

X_train=pd.DataFrame(l1,columns=X_train.columns.tolist()+tr.columns.tolist())
X_train.shape
## combine all 'one-hot' encoded features as Te.
te =pd.DataFrame(pd.np.column_stack([ te_dummy_rating_text,te_dummy_city,te_dummy_cuisines]))

## CONCAT both dataframe ### ie Te and X_test(original dataframe)
## https://stackoverflow.com/questions/45963799/pandas-concat-resulting-in-nan-rows

l3=X_test.values.tolist()
l4=te.values.tolist()
for i in range(len(l3)):
    l3[i].extend(l4[i])

X_test=pd.DataFrame(l3,columns=X_test.columns.tolist()+te.columns.tolist())
X_test.shape
# after onehot encoding DONE. 'location','rest_type','cuisines' are redundant features. REMOVE them.

X_train =X_train.drop(['city','rating_text','cuisines','review'],axis = 1)
X_test =X_test.drop(['city','rating_text','cuisines','review'],axis = 1)
X_train.head(2)
X_train.shape, X_test.shape, y_train.shape, y_test.shape
print(x.shape)
print(y.shape)
dump(X_train, 'max_features_pkl/more_feature_X_train')
dump(X_test, 'max_features_pkl/more_feature_X_test')
dump(y_train, 'max_features_pkl/more_feature_y_train')
dump(y_test, 'max_features_pkl/more_feature_y_test')
X_train= load('max_features_pkl/more_feature_X_train') 
X_test= load('max_features_pkl/more_feature_X_test')
y_train= load('max_features_pkl/more_feature_y_train')
y_test= load('max_features_pkl/more_feature_y_test')
from sklearn.metrics import  make_scorer
from sklearn.model_selection import GridSearchCV
from sklearn import metrics


def mse(y, y_pred):
    return np.mean((y_pred - y)**2) 

mse_scorer = make_scorer(mse, greater_is_better=False)<jupyter_output><empty_output><jupyter_text>Building Random Model(For Worst case MSE)<jupyter_code>import random

rand_pred= np.zeros(y_test.shape[0])
for i in range(y_test.shape[0]):
    rand_probs = round(random.uniform(1.0, 5.0),2)
    rand_pred[i] = rand_probs

mse(y_test, rand_pred)<jupyter_output><empty_output><jupyter_text>Here we can see that random model provide 2.53 MSE value. We can take this as a cut off. Any model which has MSE less than random model is considered as better model.  Model -1 Linear Regression<jupyter_code>from sklearn.linear_model import LinearRegression
lr = LinearRegression()
lr.fit(X_train,y_train)
y_pred_lr = lr.predict(X_test)

mse(y_test, y_pred_lr)<jupyter_output><empty_output><jupyter_text>Model -2 Random Forest Regressor<jupyter_code>from sklearn.ensemble import RandomForestRegressor

rfr = RandomForestRegressor()
rfr.fit(X_train,y_train)
y_pred_rfr = rfr.predict(X_test)

mse(y_test, y_pred_rfr)<jupyter_output><empty_output><jupyter_text>Feature Importance<jupyter_code>import matplotlib.pyplot as plt

#plot graph of feature importances for better visualization
feat_importances = pd.Series(rfr.feature_importances_, index=X_train.columns)
feat_importances.nlargest(10).plot(kind='barh')
plt.show()
tuned_parameters = {'n_estimators': [150]}

grd_regressor = GridSearchCV(RandomForestRegressor(), tuned_parameters, cv=10, 
                   n_jobs=-1, verbose=1, scoring=mse_scorer)
grd_regressor.fit(X_train, y_train)

y_pred_grd = grd_regressor.predict(X_test)

mse(y_test, y_pred_grd)<jupyter_output>Fitting 10 folds for each of 1 candidates, totalling 10 fits<jupyter_text>After above experiment we got below result- MSE:  0.04004986221340383 == 50(n_estimators)- MSE:  0.03807506691137555 == 100 (n_estimators)- MSE:  0.0377001297623944 == 150 (n_estimators)- MSE:  0.03828087737268492 ==200(n_estimators)- MSE:  0.03799677087809369 ===500 (n_estimators)Final n_estimators choose 150  Best Parameter Model<jupyter_code>rfr = RandomForestRegressor(max_depth=None,n_estimators=150,min_samples_split= 2)
rfr.fit(X_train,y_train)
y_pred_rfr = rfr.predict(X_test)

mse(y_test, y_pred_rfr)
Randpred = pd.DataFrame({ "actual": y_test, "pred": y_pred_rfr })
Randpred<jupyter_output><empty_output><jupyter_text>MSE = 0.0377 , It is good Model still can we still improved Model?  Model 3: Natural Language Processing 3.1 NLP on review data<jupyter_code>data_nlp = pd.read_csv('zomato_final.csv')
data_nlp.head()
import re
import nltk
from nltk.corpus import stopwords
nltk.download('stopwords')

word_counter = []
def clean_text(text):
    
    text = text.lower()    
    
    text = re.sub(r'[_"\-;%()|+&=*%.,!?:#$@\[\]/]', '', text)
    
    text = re.sub(r'>\u200b“–\\`🍅‐♥•°„', '', text)
    
    text = re.sub(r'\'', '', text)
    
    text = re.sub(r'  ', ' ', text)
    
    text = re.sub(r'\d+', '', text)    
    
    text = text.split()
    
    text =  [word for word in text if not word in set(stopwords.words("english"))]
    
    text = ' '.join(text)

    count = len(text.split())
    
    word_counter.append(count)
    
    return text

def decontracted(phrase):
    # specific
    phrase = re.sub(r"won't", "will not", phrase)
    phrase = re.sub(r"can\'t", "can not", phrase)

    # general
    phrase = re.sub(r"n\'t", " not", phrase)
    phrase = re.sub(r"\'re", " are", phrase)
    phrase = re.sub(r"\'s", " is", phrase)
    phrase = re.sub(r"\'d", " would", phrase)
    phrase = re.sub(r"\'ll", " will", phrase)
    phrase = re.sub(r"\'t", " not", phrase)
    phrase = re.sub(r"\'ve", " have", phrase)
    phrase = re.sub(r"\'m", " am", phrase)
    return phrase
data_nlp.review = data_nlp.review.apply(lambda x: clean_text(x))
data_nlp.review = data_nlp.review.apply(lambda x: decontracted(x))
data_nlp.review[1]
data_nlp['word_count'] = word_counter
# dropping unwanted columns
data_nlp = data_nlp.drop(['restaurant_id', 'restaurant_name', 'country_code', 'address','locality', 'locality_verbose', 'longitude', 'latitude', 'cuisines',
        'switch_to_orvder_menu','rating_text' ,'price_range', 'rating_color',],axis = 1)
data_nlp<jupyter_output><empty_output><jupyter_text>Exporting clean text review<jupyter_code>data_nlp.to_csv (r'preprocessed_review.csv', index = False, header=True)
from sklearn.feature_extraction.text import CountVectorizer

y = data_nlp['aggregate_rating']
X = data_nlp.drop(['aggregate_rating'],axis = 1)


# Splitting into x and y into train and test set
X_train, x_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42)
# Splitting train set into tr and cv set
x_tr, x_cv, y_tr, y_cv = train_test_split(X_train, y_train, test_size = 0.2,random_state = 42)

x_tr_txt = x_tr['review']
x_cv_txt = x_cv['review']
x_test_txt = x_test['review']

print(x_tr_txt.shape,y_tr.shape )
print(x_cv_txt.shape,y_cv.shape )
print(x_test_txt.shape,y_test.shape )

x_tr_txt = x_tr['review']
x_cv_txt = x_cv['review']
x_test_txt = x_test['review']
count_vect = CountVectorizer(ngram_range=(1,1), min_df=10) #in scikit-learn
# train data
X_train_bow = count_vect.fit_transform(x_tr_txt)

# test data
x_cv_bow = count_vect.transform(x_cv_txt)
x_test_bow = count_vect.transform(x_test_txt)


print('X_train_bow',X_train_bow.shape)
print('==='*10)
print('x_cv_bow',x_cv_bow.shape)
print('==='*10)
print('x_test_bow',x_test_bow.shape)<jupyter_output>X_train_bow (537, 856)
==============================
x_cv_bow (135, 856)
==============================
x_test_bow (168, 856)<jupyter_text>Linear regression on NLP<jupyter_code>from sklearn.linear_model import LinearRegression
lr = LinearRegression()
lr.fit(X_train_bow,y_tr)
y_pred_lr = lr.predict(x_cv_bow)

mse(y_cv, y_pred_lr)<jupyter_output><empty_output><jupyter_text>Random Forest on NLP<jupyter_code>from sklearn.ensemble import RandomForestRegressor

rfr = RandomForestRegressor()
rfr.fit(X_train_bow,y_tr)
y_pred_rfr = rfr.predict(x_cv_bow)

mse(y_cv, y_pred_rfr)<jupyter_output><empty_output><jupyter_text>observation  LSTM Model<jupyter_code>X = data['review']
y = data['rate']

# split the data set into train and test
X_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.3,random_state=0, shuffle=True)

print(X_train.shape, x_test.shape, y_train.shape, y_test.shape)

from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.layers import Input, Embedding, Flatten, LSTM, Dense, Dropout
from keras.models import Sequential, Model
from keras.utils import plot_model

max_review_length = 600
top_words = 10000

## pass through tokenizer
tk = Tokenizer(lower = True, num_words= top_words)
tk.fit_on_texts(X_train)
X_train_seq = tk.texts_to_sequences(X_train)
## add padding
X_train_pad = pad_sequences(X_train_seq, maxlen=max_review_length)
X_test_seq = tk.texts_to_sequences(x_test)
X_test_pad = pad_sequences(X_test_seq, maxlen=max_review_length)

print('train data shape ', X_train_pad.shape)
print('test data shape', X_test_pad.shape)

# create the model
embedding_vecor_length = 256
model = Sequential()
model.add(Embedding(top_words+1, embedding_vecor_length, input_length=max_review_length))
model.add(LSTM(200))  # returns a sequence of vectors of dimension 32
model.add(Dropout(0.5))
model.add(Dense(1, activation='linear'))
model.compile(loss='mean_squared_error', optimizer='adam')
print(model.summary())<jupyter_output>Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_2 (Embedding)      (None, 600, 256)          2560256   
_________________________________________________________________
lstm_2 (LSTM)                (None, 200)               365600    
_________________________________________________________________
dropout_1 (Dropout)          (None, 200)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 201       
=================================================================
Total params: 2,926,057
Trainable params: 2,926,057
Non-trainable params: 0
_________________________________________________________________
None<jupyter_text>LSTM parameter calculationembedding_vecor_length = 256top words = 10000+1 = 10001params at emdedding layers = 10001* 256 = 2560256lstm params = 4(nm + n^2 + n)where,  n = no of units; m = input dimensionshere lstm units (n) = 200 and m = 256 dimensionsparams at lstm_1: 4 (200 * 256  +  200^2  +  200) = 365600at dropout there no training happens, so params 0at dense,(input dim + 1 (bias))* no of units(200+1) * 1 = 201final trainable params,  (2560256 + 365600 + 201) = 2,926,057<jupyter_code>history = model.fit(X_train_pad, y_train, validation_data=(X_test_pad, y_test), 
                    epochs=20, 
                    verbose=1
                    )
# evaluate the model
train_mse = model.evaluate(X_train_pad, y_train, verbose=0)
test_mse = model.evaluate(X_test_pad, y_test, verbose=0)
print('Train: %.3f, Test: %.3f' % (train_mse, test_mse))
# # plot loss during training

plt.title('Loss / Mean Squared Error')
plt.plot(history.history['loss'], label='train')
plt.plot(history.history['val_loss'], label='test')
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Observation<jupyter_code># serialize model to JSON
model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model to disk")
from keras.models import model_from_json

# load json and create model
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("model.h5")
y_pred_lstm = loaded_model.predict(X_test_pad)
y_pred_lstm = y_pred_lstm.flatten()
y_pred_lstm.shape, y_test.shape
lstmpred = pd.DataFrame({ "actual": y_test, "pred": y_pred_lstm })
lstmpred<jupyter_output><empty_output><jupyter_text>4. All Features**Text data**1. review**Categorical data**1. 2.3.4.5.**Numerical data**1. 2.3.4.  Plan of attack:- Step1:  For text features, we will use pre-trained **Word2Vec** model, introduce word embedding, pass throgh LSTM layer then flatten the output.- Step2 : Similarly we will perform embedding on categorical data and flatten it.- Step3 : Merge all numerical features and scale it.- Step4 : We will concate all the feature under one block.- Step5 : pass this blob through NN, let's the output.  word 2 vec model<jupyter_code>!wget -c "https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz"
# 
import gensim
from gensim import models

word2vec = models.KeyedVectors.load_word2vec_format('GoogleNews-vectors-negative300.bin.gz', binary=True)
print('Found %s word vectors of word2vec' % len(word2vec.vocab))

# model = gensim.models.Word2Vec.load_word2vec_format('GoogleNews-vectors-negative300.bin.gz', binary=True)<jupyter_output>--2020-06-04 10:30:35--  https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz
Resolving s3.amazonaws.com (s3.amazonaws.com)... 172.16.31.10
Connecting to s3.amazonaws.com (s3.amazonaws.com)|172.16.31.10|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1647046227 (1.5G) [application/x-gzip]
Saving to: ‘GoogleNews-vectors-negative300.bin.gz’

GoogleNews-vectors- 100%[===================>]   1.53G  69.3MB/s    in 25s     

2020-06-04 10:31:01 (62.7 MB/s) - ‘GoogleNews-vectors-negative300.bin.gz’ saved [1647046227/1647046227]

Found 3000000 word vectors of word2vec<jupyter_text>Embedding layer for text data<jupyter_code>#importing clean text 
data_nlp = pd.read_csv('preprocessed_review.csv')

# seperating feature and target column
y = data_nlp['aggregate_rating']
X = data_nlp.drop(['aggregate_rating'],axis = 1)
# Define params

MAX_SEQUENCE_LENGTH = 500
MAX_NUM_WORDS = 1000
EMBEDDING_DIM = 300
## preproces

# Splitting into x and y into train and test set
X_train, x_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42)

# Splitting train set into tr and cv set
x_tr, x_cv, y_tr, y_cv = train_test_split(X_train, y_train, test_size = 0.2,random_state = 42)

x_tr_txt = x_tr['review']
x_cv_txt = x_cv['review']
x_test_txt = x_test['review']

print(x_tr_txt.shape,y_tr.shape )
print(x_cv_txt.shape,y_cv.shape )
print(x_test_txt.shape,y_test.shape )

x_tr_txt = x_tr['review']
x_cv_txt = x_cv['review']
x_test_txt = x_test['review']
## vectorize a text corpus, here we given vocab_size
tk = Tokenizer(lower = True, num_words= MAX_NUM_WORDS)

## train
tk.fit_on_texts(x_tr_txt)
X_train_seq = tk.texts_to_sequences(x_tr_txt)
X_train_pad = pad_sequences(X_train_seq, maxlen=MAX_SEQUENCE_LENGTH) 
## MAX_SEQUENCE_LENGTH = 500


## test
X_test_seq = tk.texts_to_sequences(x_test_txt)
X_test_pad = pad_sequences(X_test_seq, maxlen=MAX_SEQUENCE_LENGTH) 
## MAX_SEQUENCE_LENGTH = 500

## cv
X_cv_seq = tk.texts_to_sequences(x_cv_txt)
X_cv_pad = pad_sequences(X_cv_seq, maxlen=MAX_SEQUENCE_LENGTH) 
## MAX_SEQUENCE_LENGTH = 500

print('train data shape ', X_train_pad.shape)
print('CV data shape ', X_cv_pad.shape)
print('test data shape', X_test_pad.shape)

print('Preparing embedding matrix.')

nb_words = min(MAX_NUM_WORDS, len(tk.word_index))+1

embedding_matrix_w2v = np.zeros((MAX_NUM_WORDS, EMBEDDING_DIM))
for word, i in tk.word_index.items():
    if i >= MAX_NUM_WORDS:
        continue
      
    if word in word2vec.vocab:
        embedding_matrix_w2v[i] = word2vec.word_vec(word)
print('Null word embeddings: %d' % np.sum(np.sum(embedding_matrix_w2v, axis=1) == 0))
        
print("Number for unique words in train data:", len(tk.word_index) + 1)
print("Shape of train weight matrix:", embedding_matrix_w2v.shape)
# Creating an input layer
input_size = min(MAX_NUM_WORDS, len(tk.word_index) + 1)
input_lay = Input(shape = (MAX_SEQUENCE_LENGTH, ), name = "input_text_data")

# Creating an embedding layer
embedding_layer = Embedding(input_dim = input_size,
                            output_dim = EMBEDDING_DIM,
                            weights=[embedding_matrix_w2v],
                            input_length=MAX_SEQUENCE_LENGTH,
                            trainable=True, name = "lstm_text_data")(input_lay)
# LSTM layer
emb_lay_text = LSTM(256, return_sequences = True, dropout = 0.3)(embedding_layer)

# Flatten
flatten_1 = Flatten()(emb_lay_text)<jupyter_output><empty_output><jupyter_text>Embedding layer for Categorical Features<jupyter_code>X.info()
## city

tokenizer = Tokenizer()
tokenizer.fit_on_texts(x_tr.city)
city_tr = np.array(tokenizer.texts_to_sequences(x_tr.city))
city_cv = np.array(tokenizer.texts_to_sequences(x_cv.city))
city_ts = np.array(tokenizer.texts_to_sequences(x_test.city))

print(city_tr.shape)
print(city_cv.shape)
print(city_ts.shape)

print(tokenizer.word_index)
# Unique values
city_uni = x_tr['city'].nunique()
embedding_dim_city = 2

# Creating an input layer
input_city = Input(shape = (1,), name = "input_city")

# Creating an embedding layer
embedd_city = Embedding(input_dim = city_uni+1, output_dim = embedding_dim_city,trainable = True, 
                              name = "city_embedding_layer")(input_city)
# Flatten Book table                              
flatten_city = Flatten()(embedd_city)<jupyter_output><empty_output><jupyter_text>Embedding layer for Numerical Features<jupyter_code>## reshape the numerical features to (-1,1)

# Train data
tr_1 = x_tr['votes'].values.reshape(-1, 1)
tr_2 = x_tr['average_cost_for_two'].values.reshape(-1, 1)
tr_3 = x_tr['word_count'].values.reshape(-1, 1)
tr_4 = x_tr['has_table_booking'].values.reshape(-1, 1)
tr_5 = x_tr['has_online_delivery'].values.reshape(-1, 1)
tr_6 = x_tr['is_delivering_now'].values.reshape(-1, 1)

# CV
cv_1 = x_cv['votes'].values.reshape(-1, 1)
cv_2 = x_cv['average_cost_for_two'].values.reshape(-1, 1)
cv_3 = x_cv['word_count'].values.reshape(-1, 1)
cv_4 = x_cv['has_table_booking'].values.reshape(-1, 1)
cv_5 = x_cv['has_online_delivery'].values.reshape(-1, 1)
cv_6 = x_cv['is_delivering_now'].values.reshape(-1, 1)


# Test Data
test_1 = x_test['votes'].values.reshape(-1, 1)
test_2 = x_test['average_cost_for_two'].values.reshape(-1, 1)
test_3 = x_test['word_count'].values.reshape(-1, 1)
test_4 = x_test['has_table_booking'].values.reshape(-1, 1)
test_5 = x_test['has_online_delivery'].values.reshape(-1, 1)
test_6 = x_test['is_delivering_now'].values.reshape(-1, 1)
## Concate features

# Train
tr_num = np.concatenate((tr_1, tr_2, tr_3, tr_4, tr_5, tr_6), axis = 1)

# CV
cv_num = np.concatenate((cv_1, cv_2, cv_3, cv_4, cv_5, cv_6), axis = 1)

# Test
test_num = np.concatenate((test_1, test_2, test_3, test_4, test_5, test_6), axis = 1)
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()

tr_num = scaler.fit_transform(tr_num)

# always avoid data leakage
cv_num = scaler.transform(cv_num)
test_num = scaler.transform(test_num)
inp_num = Input(shape=(6,), name = "numerical_features")
emb_num = Dense(128, activation = "relu")(inp_num)
from keras.layers import concatenate

# Concatenating all
con_lay = concatenate([flatten_1,flatten_city, emb_num])
# Dense Layer 1
DL1 = Dense(896, activation = 'relu')(con_lay)

# Dropout 1
DP1 = Dropout(0.5)(DL1)

# Output
output = Dense(1, activation = 'linear', name= 'model_1_output')(DP1)

# READ THIS
# Model
# rename input_lay = input_text
# rename input_city = input_categorical
# rename input_numerical = input_numerical
# 
model1 = Model(inputs = [input_lay,input_city,inp_num], outputs = output)

plot_model(model1, show_shapes=True,show_layer_names=True, to_file='model1.png')
# Train data
tr_data_1 = [X_train_pad, city_tr,  tr_num ]

# CV data
cv_data_1 = [X_cv_pad, city_cv,  cv_num]

# Test data
test_data_1 = [X_test_pad, city_ts, test_num]

model1.summary()<jupyter_output>Model: "model_5"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_text_data (InputLayer)    (None, 500)          0                                            
__________________________________________________________________________________________________
lstm_text_data (Embedding)      (None, 500, 300)     300000      input_text_data[0][0]            
__________________________________________________________________________________________________
input_city (InputLayer)         (None, 1)            0                                            
__________________________________________________________________________________________________
lstm_5 (LSTM)                   (None, 500, 256)     570368      lstm_text_data[0][0]       [...]<jupyter_text>Neural network param calculations,at LSTM layer = 4 (256 * 300 + 256 ^ 2 + 256 ) = 570368at dense_9 layer = (6+1(bias)) * 128 = 896 at dense_18 layer = (128132 + 1 (bias) ) * 512 = 114807168 at model_1_output = (896+1) * 1 = 897 total params adding all = 300000 + 570368 + 896 + 114807168 + 897 = 115677537 --><jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12779080600083523, 'fraction_numerical': 0.028911947059012497, 'mean_word_length': 5.43846949327818, 'pattern_counts': {'":': 4, '<': 107, '<?xml version=': 0, '>': 117, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 3, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '28', 'dir': 'jupyter-structured-clean-dedup', 'id': '466307', 'license': 'MIT', 'n_tokens_mistral': 11579, 'n_tokens_neox': 10854, 'n_words': 2381, 'path': 'research/002_exploratory_data_analysis.ipynb', 'repo_name': 'ashish-gh/Exploratory_Data_Analysis_Zomato_Restaurant', 'size': '31131'}