<jupyter_start><jupyter_text>Amazon Food Reviews Recurrent Neural Network<jupyter_code>pwd<jupyter_output><empty_output><jupyter_text>Algorithm and Deep LearningIn order to predict if a review is positive or negative we will use a recurrent neural network. Most of the methods for sentiment analysis look at each word individually, attribute positive points for positive words and negative points for negative words, and then total the points. This is the lexicon approach. However the problem with this method is that it ignores the sequence of the words, which can lead to the loss of important information. The RNN approach can understand subtleties because it doesn’t analyze text at face value. It creates abstract representations of what it learned.We will use Long Short Term Memory Networks (LSTM), which is a special case of RNN. The main advantage of LSTM is that it can memorize information. When there's too large of a gap between two pieces of information, RNN is unable to learn to connect information.<jupyter_code>import pandas as pd
import numpy as np
import pickle
import matplotlib.pyplot as plt
from scipy import stats
import tensorflow as tf
import seaborn as sns
from pylab import rcParams
from sklearn.model_selection import train_test_split
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers import Input, Dense, LSTM
from keras.callbacks import ModelCheckpoint, TensorBoard
from keras import regularizers
%matplotlib inline
from keras.layers.core import Flatten
from keras.layers.core import Dense

sns.set(style='whitegrid', palette='muted', font_scale=1.5)
rcParams['figure.figsize'] = 14, 8
# Feature Scaling
from sklearn.preprocessing import MinMaxScaler

mns = MinMaxScaler()
# Initialising the RNN
regressor = Sequential()
data = pd.read_csv("FoodReview.csv")
data.head()
data.columns
# split table into training table
training_table = data[['Id','HelpfulnessNumerator','HelpfulnessDenominator', 'Prediction', 'Time']]<jupyter_output><empty_output><jupyter_text>Output: positive or negative?Finally, we will predict if the review is positive or negative using the sigmoid function. The advantage of the function is that it's bound between 0 and 1 and can be interpreted as a probability of success. For example, we can estimate the probability that a review is positive. At each step, we have an output. However, we only care about the final output that predicts the sentiment at the end of the sentence.<jupyter_code>training_table.head()<jupyter_output><empty_output><jupyter_text>Feature Scaling<jupyter_code>training_table = training_table.iloc[:, 2:3].values

# Feature Scaling
from sklearn.preprocessing import MinMaxScaler
mns = MinMaxScaler()
training_table = mns.fit_transform(training_table)<jupyter_output>/Users/user/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by MinMaxScaler.
  warnings.warn(msg, DataConversionWarning)<jupyter_text>Tuning hyperparametersFor our recurrent neural network we will have to tune the following hyperparameters:Optimizer hyperparameters:1.) Learning rate2.) Minibatch size3.) Number of epochsModel hyperparameters:1.) Number of hidden layers (LSTM layers)2.) Number of units in the LSTM cells3.) Dropout rate<jupyter_code># Getting the inputs and the ouputs
x_train =  training_table[0:999]
y_train = training_table[1:1000]

# Reshaping
x_train = x_train.reshape(999,1,1)
print(x_train.shape)
print(y_train.shape)<jupyter_output>(999, 1, 1)
(999, 1)<jupyter_text>Generate LSTM  Long Short Term MemoryThe next step is to pass the embedded words to the LSTM cells. In the graph, you can observe that there are connections between the first, second and third LSTM cells. In contrast to other models that assume that all the inputs are independent of each other, these connections here allow us to pass information contained in the sequence of words. One of the strengths of the LSTM cell is the ability to add or remove information to the cell state.<jupyter_code>#LSTM
regressor.add(LSTM(12, activation='relu', input_shape=(None,1)))
regressor.add(Dense(8, activation='softmax'))
regressor.add(Dense(1))
regressor.compile(optimizer='adadelta', loss='mean_squared_error')
# Fitting the RNN to the Training set
regressor.fit(x_train, y_train, batch_size=20, epochs=20)<jupyter_output>Epoch 1/20
999/999 [==============================] - 1s 885us/step - loss: 5.0509e-04
Epoch 2/20
999/999 [==============================] - 0s 489us/step - loss: 2.7751e-05
Epoch 3/20
999/999 [==============================] - 0s 392us/step - loss: 2.7957e-05
Epoch 4/20
999/999 [==============================] - 0s 428us/step - loss: 2.8000e-05
Epoch 5/20
999/999 [==============================] - 0s 411us/step - loss: 2.8153e-05
Epoch 6/20
999/999 [==============================] - 0s 414us/step - loss: 2.8068e-05
Epoch 7/20
999/999 [==============================] - 0s 411us/step - loss: 2.8173e-05
Epoch 8/20
999/999 [==============================] - 0s 420us/step - loss: 2.8092e-05
Epoch 9/20
999/999 [==============================] - 0s 422us/step - loss: 2.8661e-05
Epoch 10/20
999/999 [==============================] - 0s 421us/step - loss: 2.8412e-05
Epoch 11/20
999/999 [==============================] - 0s 421us/step - loss: 2.8253e-05
Epoch 12/20
999/999 [====================[...]<jupyter_text>Simple RNN<jupyter_code>from keras.layers.recurrent import SimpleRNN
from keras.layers import Dropout<jupyter_output><empty_output><jupyter_text>To train the RRN on the review dataset we have to create an array with 200 columns, and enough rows to fit one review per row. Then store each review consisting of integers in the array. If the number of words in the review is less than 200 words, we can pad the list with extra zeros. If the number of words in the review are more than 200 words then we can limit the review to the first 200 words.<jupyter_code># Adding the input layer and the LSTM layer
model = Sequential()
model.add(SimpleRNN(128,input_shape = (None,1)))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid',kernel_initializer='uniform'))
model.compile(optimizer='adam', loss='binary_crossentropy')
y_train = training_table[1:1000]
model.fit(x_train, y_train, batch_size=8, epochs=100)
# evaluate the model
scores = model.evaluate(x_train, y_train)
print("\n%s: %.2f%%" % (model.metrics_names, scores*100))<jupyter_output>999/999 [==============================] - 0s 211us/step

['loss']: 1.53%
{'fraction_non_alphanumeric': 0.1375726077652094, 'fraction_numerical': 0.05105472332619994, 'mean_word_length': 5.836990595611285, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '376902', 'license': 'MIT', 'n_tokens_mistral': 2200, 'n_tokens_neox': 1915, 'n_words': 718, 'path': 'INFO7390_FinalProject_RNN_AmazonFoodReview.ipynb', 'repo_name': 'spurthishetty/Sentimental-Analysis-on-Amazon-Food-Reviews', 'size': '6542'}