<jupyter_start><jupyter_text>RNN - LSTM - Toxic CommentsA corpus of manually labeled comments - classifying each comment by its type of toxicity is available on Kaggle. We will aim to do a binary classification of whether a comment is toxic or not.Approach:- Learning Embedding with the Task- LSTM- BiLSTM<jupyter_code>import numpy as np
import pandas as pd
import keras
import matplotlib.pyplot as plt
%matplotlib inline
import vis<jupyter_output>Using TensorFlow backend.<jupyter_text>Get the Data Uncomment these shell lines to get the data<jupyter_code># !wget http://bit.do/deep_toxic_train -P data/
# !mv data/deep_toxic_train data/train.zip
df = pd.read_csv("data/train.zip")
df.head()<jupyter_output><empty_output><jupyter_text>Import the required libraries<jupyter_code>from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences<jupyter_output><empty_output><jupyter_text>Create the Input & Output Data<jupyter_code>train_sentences = df["comment_text"]
train_sentences.head()<jupyter_output><empty_output><jupyter_text>**Pre-processing the train data**- Tokenization: "This is an apple" -> ["This", "is", "an", "apple"]- Indexing: {0: "This", 1: "is", 2: "an", 3: "apple"}- Index Representation: [0, 1, 2, 3]<jupyter_code>from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
# Tokenizer
max_words = 20000
tokenizer = Tokenizer(num_words=max_words, oov_token='UNK')
tokenizer.fit_on_texts(list(train_sentences))<jupyter_output><empty_output><jupyter_text>Tokenizer Fix from https://github.com/keras-team/keras/issues/8092<jupyter_code>tokenizer.word_index = {e:i for e,i in tokenizer.word_index.items() if i <= max_words} # <= because tokenizer is 1 indexed
tokenizer.word_index[tokenizer.oov_token] = max_words + 1
# Index Representation
tokenized_train = tokenizer.texts_to_sequences(train_sentences)
# Selecting Padding
# find length of each sentence and plot the length
number_of_words = [len(comment) for comment in tokenized_train]
plt.hist(number_of_words, bins = np.arange(0, 500, 10));
# Padding to make it uniform
maxlen = 200
X = pad_sequences(tokenized_train, maxlen = maxlen)
labels = df.iloc[:,2].values
# Baseline Benchmark 
1 - df.iloc[:,2].sum()/df.iloc[:,2].count()
from keras.utils import to_categorical
y = to_categorical(labels)
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
X_train.shape, X_test.shape, y_train.shape, y_test.shape<jupyter_output><empty_output><jupyter_text>Step 2: Create the Model Architecture<jupyter_code>from keras.models import Sequential
from keras.layers import Dense, LSTM, Embedding, Dropout, Bidirectional
model = Sequential()
model.add(Embedding(max_words, output_dim=128, mask_zero=True))
model.add(LSTM(60))
model.add(Dropout(0.1))
model.add(Dense(2, activation='sigmoid'))<jupyter_output><empty_output><jupyter_text>Step 3: Compile the Model & Fit on the Data<jupyter_code>model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
output = model.fit(X_train, y_train, batch_size=128, epochs=5, validation_split=0.2)<jupyter_output>Train on 102124 samples, validate on 25532 samples
Epoch 1/5
102124/102124 [==============================] - 398s 4ms/step - loss: 0.1625 - acc: 0.9454 - val_loss: 0.1185 - val_acc: 0.9601
Epoch 2/5
102124/102124 [==============================] - 352s 3ms/step - loss: 0.1096 - acc: 0.9618 - val_loss: 0.1050 - val_acc: 0.9629
Epoch 3/5
102124/102124 [==============================] - 436s 4ms/step - loss: 0.1006 - acc: 0.9642 - val_loss: 0.1022 - val_acc: 0.9642
Epoch 4/5
102124/102124 [==============================] - 333s 3ms/step - loss: 0.0954 - acc: 0.9654 - val_loss: 0.1071 - val_acc: 0.9635
Epoch 5/5
102124/102124 [==============================] - 329s 3ms/step - loss: 0.0923 - acc: 0.9665 - val_loss: 0.1026 - val_acc: 0.9640<jupyter_text>Step 4: Evaluate the Model<jupyter_code>vis.metrics(output.history)
score = model.evaluate(X_test, y_test, verbose=1)
print('Test loss:', score[0])
print('Test accuracy:', score[1])<jupyter_output>Test loss: 0.10434365738769805
Test accuracy: 0.9634654551151496<jupyter_text>Step 5: Visualise evaluation & Make a prediction<jupyter_code>predict_classes = model.predict_classes(X_test)
actual_classes = np.dot(y_test,np.array([[0],[1]])).reshape(-1)
pd.crosstab(actual_classes, predict_classes)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15055679287305124, 'fraction_numerical': 0.06592427616926504, 'mean_word_length': 6.650766609880749, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 39, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '3183', 'license': 'MIT', 'n_tokens_mistral': 1806, 'n_tokens_neox': 1585, 'n_words': 367, 'path': '011-RNN-LSTM-Toxic.ipynb', 'repo_name': 'SaketJNU/deep-learning', 'size': '4490'}