<jupyter_start><jupyter_text>In this Notebook we'll be developing a machine learning model to assess if a Twitter post related to vaccinations is positive, neutral, or negative. This solution could help governments and other public health actors monitor public sentiment towards COVID-19 vaccinations and help improve public health policy, vaccine communication strategies, and vaccination programs across the world.
We will start very simple to understand the general concepts whilst not really caring about good results.  Load Libraries<jupyter_code>import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import warnings
warnings.filterwarnings('ignore')
import seaborn as sns
from sklearn.model_selection import train_test_split
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
import tensorflow as tf
import time 
import re
# Natural Language Tool Kit 
import nltk  
nltk.download('stopwords') 
from nltk.corpus import stopwords<jupyter_output>[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Unzipping corpora/stopwords.zip.<jupyter_text>2. Load Data<jupyter_code>train = pd.read_csv("/content/Train.csv")
test = pd.read_csv("/Test.csv")
train.head()
test.head()
train['label'].value_counts()<jupyter_output><empty_output><jupyter_text>2. Cleaning Data
- ensuring it is a string
- converting capital letters to small letters
- removing characters and replacing them with space
- removing  of punctuations marks<jupyter_code>train['safe_text'] = train['safe_text'].apply(str)
test['safe_text'] = test['safe_text'].apply(str)
train['safe_text'] = train['safe_text'].apply(str.lower)
test['safe_text'] = test['safe_text'].apply(str.lower)
train['safe_text'] = train['safe_text'].apply(lambda x: x.replace('&amp;', ' '))
test['safe_text'] = test['safe_text'].apply(lambda x: x.replace('&amp;', ' '))
train['safe_text'] = train['safe_text'].apply(lambda x: x.replace('<user>', ' '))
test['safe_text'] = test['safe_text'].apply(lambda x: x.replace('<user>', ' '))
train['safe_text'] = train['safe_text'].apply(lambda x: x.replace('<url>', ' '))
test['safe_text'] = test['safe_text'].apply(lambda x: x.replace('<url>', ' '))
train['safe_text'] = train['safe_text'].apply(lambda x: x.replace('#', ' '))
test['safe_text'] = test['safe_text'].apply(lambda x: x.replace('#', ' '))
train['safe_text'] = train['safe_text'].apply(lambda x: x.strip('.').strip())
test['safe_text'] = test['safe_text'].apply(lambda x: x.strip('.').strip())
train["label"].unique()<jupyter_output><empty_output><jupyter_text>we only have one row with label 0.6667 and no agreement, so we would be dropping the row<jupyter_code>train.drop(index=[4798, 4799], inplace=True)
train.reset_index(drop=True, inplace=True)
train.dropna(inplace=True)
train["label"].unique()
train.head()
# max number of words in each sentence
SEQUENCE_LENGTH = 50
EMBEDDING_SIZE = 100
# number of words to use, discarding the rest
N_WORDS = 10000
# out of vocabulary token
OOV_TOKEN = "<unk>"
padding_type = 'post'
trunc_type = 'post'<jupyter_output><empty_output><jupyter_text>3. Preprocessing Data
Link to Learn More About the steps taken below
https://www.kdnuggets.com/2020/03/tensorflow-keras-tokenization-text-data-prep.html - Our Labels are within -1 to 1, it should not be less than 1, so we + 1 
- Convert Labels for a multi classification task using to categorical
- split our data
- We Instantiate our tokenizer class to tokenize training data text
- we pad our tokenized text to ensure  equal lengths<jupyter_code>train['label'].value_counts()
X = train['safe_text']
y = to_categorical(train['label']+1,num_classes=3)
X_train,X_val,y_train,y_val = train_test_split(X,y,test_size=0.2)
print('The Shape of Xraining ',X_train.shape)
print('The Shape of Validation',X_val.shape)
tokenizer = Tokenizer(num_words=N_WORDS, oov_token=OOV_TOKEN)
tokenizer.fit_on_texts(X_train)
word_index = tokenizer.word_index
print("THe first word Index are: ")
for x in list(word_index)[0:15]:
    print (" {},  {} ".format(x,  word_index[x]))
training_sequences = tokenizer.texts_to_sequences(X_train)
training_padded = pad_sequences(training_sequences, maxlen=SEQUENCE_LENGTH, padding=padding_type, truncating=trunc_type)
print(train.safe_text[1])
print(training_sequences[1])
test['safe_text'] = test['safe_text'].astype(str)
val_sequences = tokenizer.texts_to_sequences(X_val)
val_padded = pad_sequences(val_sequences, maxlen=SEQUENCE_LENGTH, padding=padding_type, truncating=trunc_type)
test_sequences = tokenizer.texts_to_sequences(test['safe_text'])
test_padded = pad_sequences(test_sequences, maxlen=SEQUENCE_LENGTH, padding=padding_type, truncating=trunc_type)
train.isnull().any().sum()<jupyter_output><empty_output><jupyter_text>4. Creating the Model Our four layers are an embedding layer, our LSTM, and two linear layers.<jupyter_code>optimizer = tf.keras.optimizers.Adam(lr=0.01)

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(N_WORDS, EMBEDDING_SIZE, input_length=SEQUENCE_LENGTH),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),
    tf.keras.layers.Dense(14, activation='relu'),
    tf.keras.layers.Dense(3,activation='sigmoid')
])
model.compile(loss='binary_crossentropy',optimizer=optimizer)
start_time = time.time()

num_epochs = 5
history = model.fit(training_padded, y_train, epochs=num_epochs, validation_data=(val_padded, y_val))
pred = model.predict(test_padded)
pred<jupyter_output><empty_output><jupyter_text>Important Information

we can go ahead and do `test['label'] = np.argmax(pred,axis=1)` for a normal multi classification task, with classification metrics  `accuracy`, but the metric given for the challenge is a root mean squared error metrics hence performance on leaderboard would be bad, so we preprocess our predictions to have values between -1 and 1, better preprocessing can be done to improve scores on leaderboard.<jupyter_code>def process_prediction(preds):
  r'''
    This function helps us go from a classifiaction
    problem to a regression one.
    The regression values range are in [-1, 1].
  '''

  final_preds = []
  for pred in preds:
    argmax = np.argmax(pred, axis=0)
    if argmax == 0: final_preds.append( -1*pred[0] )
    elif argmax == 1: final_preds.append( 0 )
    else: final_preds.append( pred[2] )
    
  return final_preds


def rmse(true, pred):
  return np.sqrt(mean_squared_error(true, pred))
predictions = process_prediction(pred)
test['label'] = predictions
test.head()
test[['tweet_id','label']].to_csv('submissions.csv',index=False)<jupyter_output><empty_output><jupyter_text>Some useful Insights
- The Text Cleaning done here is vanilla, you can do a lot them
- use a pretrained word embeddings e.g **Glove** 
- Try out the Transformer Architecture, or Use a Pretrained Transformer Architecture
- Tweak Neural Network Parameters
- increase no of epoch<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12164252566446351, 'fraction_numerical': 0.009703276613697088, 'mean_word_length': 6.041584158415842, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '170942', 'license': 'Apache-2.0', 'n_tokens_mistral': 2391, 'n_tokens_neox': 2313, 'n_words': 596, 'path': 'vacinate_zindi/Simple_LSTM_Zindi_To_Vaccinate_or_Not_To_Vaccinate.ipynb', 'repo_name': 'Abayomi2020-data/To-Vaccinate-or-Not-to-Vaccinate', 'size': '7111'}