<jupyter_start><jupyter_text>![alt text](https://drive.google.com/uc?export=view&id=1UXScsVx_Wni_JuDdB8LeTnM6jsPfIwkW)Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.  Sentiment Classification  Dataset- Dataset of 50,000 movie reviews from IMDB, labeled by sentiment positive (1) or negative (0)- Reviews have been preprocessed, and each review is encoded as a sequence of word indexes (integers).- For convenience, words are indexed by overall frequency in the dataset, so that for instance the integer "3" encodes the 3rd most frequent word in the data. This allows for quick filtering operations such as: "only consider the top 10,000 most common words, but eliminate the top 20 most common words".- As a convention, "0" does not stand for a specific word, but instead is used to encode any unknown word.Command to import data- `from tensorflow.keras.datasets import imdb` We start by importing the required dependencies to preprocess our data and build our model.<jupyter_code>from tensorflow.keras.datasets import imdb
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt

import numpy as np

from keras.utils import to_categorical
from keras import models
from keras import layers<jupyter_output><empty_output><jupyter_text>Import the data (2 Marks)- Use `imdb.load_data()` method- Get train and test set- Take 10000 most frequent words<jupyter_code>from keras.datasets import imdb
(training_data, training_targets), (testing_data, testing_targets) = imdb.load_data(num_words=10000)
data = np.concatenate((training_data, testing_data), axis=0)
targets = np.concatenate((training_targets, testing_targets), axis=0)
print("Categories:", np.unique(targets))
print("Number of unique words:", len(np.unique(np.hstack(data))))<jupyter_output>Categories: [0 1]
Number of unique words: 9998<jupyter_text>Pad each sentence to be of same length (2 Marks)<jupyter_code>import re
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

data = data[data.sentiment != "Neutral"]
data['text'] = data['text'].apply(lambda x: x.lower())
data['text'] = data['text'].apply((lambda x: re.sub('[^a-zA-z0-9\s]','',x)))

print(data[ data['sentiment'] == 'Positive'].size)
print(data[ data['sentiment'] == 'Negative'].size)

for idx,row in data.iterrows():
    row[0] = row[0].replace('rt',' ')
    
vocabSize = 2000
tokenizer = Tokenizer(num_words=vocabSize, split=' ')
tokenizer.fit_on_texts(data['text'].values)
X = tokenizer.texts_to_sequences(data['text'].values)
print(X)
X = pad_sequences(X)
print('pad')
print(X)<jupyter_output><empty_output><jupyter_text>* Average Review Length & Standard Deviation<jupyter_code>length = [len(i) for i in data]
print("Average Review length:", np.mean(length))
print("Standard Deviation:", round(np.std(length)))<jupyter_output>Average Review length: 234.75892
Standard Deviation: 173.0<jupyter_text>Print shape of features & labels (2 Marks) Number of review, number of words in each review<jupyter_code>data.shape
data.size
data<jupyter_output><empty_output><jupyter_text>Print value of any one feature and it's label (2 Marks) Feature value<jupyter_code>print(data[0])<jupyter_output>[1, 14, 22, 16, 43, 530, 973, 1622, 1385, 65, 458, 4468, 66, 3941, 4, 173, 36, 256, 5, 25, 100, 43, 838, 112, 50, 670, 2, 9, 35, 480, 284, 5, 150, 4, 172, 112, 167, 2, 336, 385, 39, 4, 172, 4536, 1111, 17, 546, 38, 13, 447, 4, 192, 50, 16, 6, 147, 2025, 19, 14, 22, 4, 1920, 4613, 469, 4, 22, 71, 87, 12, 16, 43, 530, 38, 76, 15, 13, 1247, 4, 22, 17, 515, 17, 12, 16, 626, 18, 2, 5, 62, 386, 12, 8, 316, 8, 106, 5, 4, 2223, 5244, 16, 480, 66, 3785, 33, 4, 130, 12, 16, 38, 619, 5, 25, 124, 51, 36, 135, 48, 25, 1415, 33, 6, 22, 12, 215, 28, 77, 52, 5, 14, 407, 16, 82, 2, 8, 4, 107, 117, 5952, 15, 256, 4, 2, 7, 3766, 5, 723, 36, 71, 43, 530, 476, 26, 400, 317, 46, 7, 4, 2, 1029, 13, 104, 88, 4, 381, 15, 297, 98, 32, 2071, 56, 26, 141, 6, 194, 7486, 18, 4, 226, 22, 21, 134, 476, 26, 480, 5, 144, 30, 5535, 18, 51, 36, 28, 224, 92, 25, 104, 4, 226, 65, 16, 38, 1334, 88, 12, 16, 283, 5, 16, 4472, 113, 103, 32, 15, 16, 5345, 19, 178, 32]<jupyter_text>Label value<jupyter_code>print("Label:", targets[0])<jupyter_output>Label: 1<jupyter_text>Decode the feature value to get original sentence (2 Marks) First, retrieve a dictionary that contains mapping of words to their index in the IMDB dataset<jupyter_code>index = imdb.get_word_index()
reverse_index = dict([(value, key) for (key, value) in index.items()]) 
decoded = " ".join( [reverse_index.get(i - 3, "#") for i in data[0]] )
print(decoded)<jupyter_output>Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json
1646592/1641221 [==============================] - 0s 0us/step
# this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert # is an amazing actor and now the same being director # father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for # and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also # to the two little boy's that played the # of norman and paul they were just brilliant children are often left out of the # list i think because the [...]<jupyter_text><jupyter_code>def vectorize(sequences, dimension = 10000):
  results = np.zeros((len(sequences), dimension))
  for i, sequence in enumerate(sequences):
    results[i, sequence] = 1
    return results
 
  data = vectorize(data)
  targets = np.array(targets).astype("float32")<jupyter_output><empty_output><jupyter_text>Define model (10 Marks)- Define a Sequential Model- Add Embedding layer  - Embedding layer turns positive integers into dense vectors of fixed size  - `tensorflow.keras` embedding layer doesn't require us to onehot encode our words, instead we have to give each word a unique integer number as an id. For the imdb dataset we've loaded this has already been done, but if this wasn't the case we could use sklearn LabelEncoder.  - Size of the vocabulary will be 10000  - Give dimension of the dense embedding as 100  - Length of input sequences should be 300- Add LSTM layer  - Pass value in `return_sequences` as True- Add a `TimeDistributed` layer with 100 Dense neurons- Add Flatten layer- Add Dense layer<jupyter_code>import numpy as np
from keras.utils import to_categorical
from keras import models
from keras import layers
from keras.datasets import imdb

(training_data, training_targets), (testing_data, testing_targets) = imdb.load_data(num_words=10000)
data = np.concatenate((training_data, testing_data), axis=0)
targets = np.concatenate((training_targets, testing_targets), axis=0)
def vectorize(sequences, dimension = 10000):
 results = np.zeros((len(sequences), dimension))
 for i, sequence in enumerate(sequences):
  results[i, sequence] = 1
 return results
data = vectorize(data)
targets = np.array(targets).astype("float32")
test_x = data[:10000]
test_y = targets[:10000]
train_x = data[10000:]
train_y = targets[10000:]
print(train_x.shape,train_y.shape)
print(test_x.shape,test_y.shape)
model = models.Sequential()
# Input - Layer
model.add(layers.Dense(50, activation = "relu", input_shape=(10000, )))
# Hidden - Layers
model.add(layers.Dropout(0.3, noise_shape=None, seed=None))
model.add(layers.Dense(50, activation = "relu"))
model.add(layers.Dropout(0.2, noise_shape=None, seed=None))
model.add(layers.Dense(50, activation = "relu"))
# Output- Layer
model.add(layers.Dense(1, activation = "sigmoid"))<jupyter_output><empty_output><jupyter_text>Compile the model (2 Marks)- Use Optimizer as Adam- Use Binary Crossentropy as loss- Use Accuracy as metrics<jupyter_code># compiling the model
model.compile(
 optimizer = "adam",
 loss = "binary_crossentropy",
 metrics = ["accuracy"]
)<jupyter_output><empty_output><jupyter_text>Print model summary (2 Marks)<jupyter_code>model.summary()<jupyter_output>Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_12 (Dense)             (None, 50)                500050    
_________________________________________________________________
dropout_6 (Dropout)          (None, 50)                0         
_________________________________________________________________
dense_13 (Dense)             (None, 50)                2550      
_________________________________________________________________
dropout_7 (Dropout)          (None, 50)                0         
_________________________________________________________________
dense_14 (Dense)             (None, 50)                2550      
_________________________________________________________________
dense_15 (Dense)             (None, 1)                 51        
======================================================[...]<jupyter_text>Fit the model (2 Marks)<jupyter_code>results = model.fit(
 train_x, train_y,
 epochs= 10,
 batch_size = 500,
 validation_data = (test_x, test_y)
)<jupyter_output>Epoch 1/10
80/80 [==============================] - 1s 17ms/step - loss: 0.3949 - accuracy: 0.8288 - val_loss: 0.2645 - val_accuracy: 0.8919
Epoch 2/10
80/80 [==============================] - 1s 14ms/step - loss: 0.2116 - accuracy: 0.9198 - val_loss: 0.2616 - val_accuracy: 0.8966
Epoch 3/10
80/80 [==============================] - 1s 14ms/step - loss: 0.1508 - accuracy: 0.9454 - val_loss: 0.2844 - val_accuracy: 0.8897
Epoch 4/10
80/80 [==============================] - 1s 14ms/step - loss: 0.1080 - accuracy: 0.9623 - val_loss: 0.3587 - val_accuracy: 0.8870
Epoch 5/10
80/80 [==============================] - 1s 13ms/step - loss: 0.0714 - accuracy: 0.9761 - val_loss: 0.4268 - val_accuracy: 0.8858
Epoch 6/10
80/80 [==============================] - 1s 13ms/step - loss: 0.0513 - accuracy: 0.9833 - val_loss: 0.4749 - val_accuracy: 0.8835
Epoch 7/10
80/80 [==============================] - 1s 13ms/step - loss: 0.0418 - accuracy: 0.9856 - val_loss: 0.5198 - val_accuracy: 0.8830
Epoch 8/10
80[...]<jupyter_text>Evaluate model (2 Marks)<jupyter_code>batch_size = 32
score,acc = model.evaluate(test_x, test_y, verbose = 2, batch_size = batch_size)
print("score: %.2f" % (score))
print("acc: %.2f" % (acc))<jupyter_output>score: 0.62
acc: 0.88<jupyter_text>Predict on one sample (2 Marks)<jupyter_code>twt = ['He is a lazy person.']
#vectorizing the tweet by the pre-fitted tokenizer instance
twt = tokenizer.texts_to_sequences(twt)
#padding the tweet to have exactly the same shape as `embedding_2` input
twt = pad_sequences(twt, maxlen=28, dtype='int32', value=0)
print(twt)
sentiment = model.predict(twt,batch_size=1,verbose = 2)
if(np.argmax(sentiment) == 0):
    print("negative")
elif (np.argmax(sentiment) == 1):
    print("positive")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14422480276571226, 'fraction_numerical': 0.08403510327098661, 'mean_word_length': 4.975635593220339, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '156902', 'license': 'MIT', 'n_tokens_mistral': 4447, 'n_tokens_neox': 3558, 'n_words': 1215, 'path': 'Questions_Project_1_Sequential_Models_in_NLP_Sentiment_Classification.ipynb', 'repo_name': 'PranilS1120/Face-Mask-Detection', 'size': '11281'}