<jupyter_start><jupyter_text>Text classification and sentiment analysis: Twitter Once text data has been converted into numerical features using the natural language processing techniques discussed in the previous sections, text classification works just like any other classification task.In this notebook, we will apply these preprocessing technique to news articles, product reviews, and Twitter data and teach various classifiers to predict discrete news categories, review scores, and sentiment polarity.  Imports<jupyter_code>import warnings
warnings.filterwarnings('ignore')
%matplotlib inline

from pathlib import Path
import numpy as np
import pandas as pd

# Visualization
import matplotlib.pyplot as plt
import seaborn as sns

# spacy, textblob and nltk for language processing
from textblob import TextBlob

# sklearn for feature extraction & modeling
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import roc_auc_score, roc_curve, accuracy_score

sns.set_style('white')<jupyter_output><empty_output><jupyter_text>Twitter Sentiment  Download the data We use a dataset that contains 1.6 million training and 350 test tweets from 2009 with algorithmically assigned binary positive and negative sentiment scores that are fairly evenly split. Follow the [instructions](../data/twitter_sentiment.ipynb) to create the dataset. - 0 - the polarity of the tweet (0 = negative, 2 = neutral, 4 = positive); training data has no neutral tweets- 1 - the id of the tweet (2087)- 2 - the date of the tweet (Sat May 16 23:58:44 UTC 2009)- 3 - the query (lyx). If there is no query, then this value is NO_QUERY. (only test data uses query)- 4 - the user that tweeted (robotickilldozr)- 5 - the text of the tweet (Lyx is cool)  Read and preprocess train/test data<jupyter_code>data_path = Path('..', 'data', 'sentiment140')
if not data_path.exists():
    data_path.mkdir(parents=True)
names = ['polarity', 'id', 'date', 'query', 'user', 'text']<jupyter_output><empty_output><jupyter_text>Take a few preprocessing steps:- remove tweets above the legal (at the time) length of 140 characters,- binarize polarity, and - move the data to the faster parquet format.<jupyter_code>def load_train_data():
    parquet_file = data_path / 'train.parquet'
    if not parquet_file.exists():
        df = (pd.read_csv(data_path / 'train.csv',
                          low_memory=False,
                          encoding='latin1',
                          header=None,
                          names=names,
                          parse_dates=['date'])
              .drop(['id', 'query'], axis=1)
              .drop_duplicates(subset=['polarity', 'text']))
        df = df[df.text.str.len() <= 140]
        df.polarity = (df.polarity > 0).astype(int)
        df.to_parquet(parquet_file)
        return df
    else:
        return pd.read_parquet(parquet_file)
train = load_train_data()
train.info(null_counts=True)
def load_test_data():
    parquet_file = data_path / 'test.parquet'
    if not parquet_file.exists():
        df = (pd.read_csv('data/sentiment140/test.csv',
                          low_memory=False,
                          encoding='latin1',
                          header=None,
                          names=names,
                          parse_dates=['date'])
              .drop(['id', 'query'], axis=1)
              .drop_duplicates(subset=['polarity', 'text']))
        df = df[(df.text.str.len() <= 140) &
                (df.polarity.isin([0, 4]))]
        df.to_parquet(parquet_file)
        return df
    else:
        return pd.read_parquet(parquet_file)
test = load_test_data()
test.info(null_counts=True)<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 354 entries, 0 to 497
Data columns (total 4 columns):
 #   Column    Non-Null Count  Dtype              
---  ------    --------------  -----              
 0   polarity  354 non-null    int64              
 1   date      354 non-null    datetime64[ns, UTC]
 2   user      354 non-null    object             
 3   text      354 non-null    object             
dtypes: datetime64[ns, UTC](1), int64(1), object(2)
memory usage: 13.8+ KB<jupyter_text>Explore data<jupyter_code>train.head()
train.polarity = (train.polarity>0).astype(int)
train.polarity.value_counts()
test.polarity = (test.polarity>0).astype(int)
test.polarity.value_counts()
sns.distplot(train.text.str.len(), kde=False)
sns.despine();
train.date.describe()
train.user.nunique()
train.user.value_counts().head(10)<jupyter_output><empty_output><jupyter_text>Create text vectorizer We create a document-term matrix with 934 tokens as follows:<jupyter_code>vectorizer = CountVectorizer(min_df=.001, max_df=.8, stop_words='english')
train_dtm = vectorizer.fit_transform(train.text)
train_dtm
test_dtm = vectorizer.transform(test.text)<jupyter_output><empty_output><jupyter_text>Train Naive Bayes Classifier<jupyter_code>nb = MultinomialNB()
nb.fit(train_dtm, train.polarity)<jupyter_output><empty_output><jupyter_text>Predict Test Polarity<jupyter_code>predicted_polarity = nb.predict(test_dtm)<jupyter_output><empty_output><jupyter_text>Evaluate Results<jupyter_code>accuracy_score(test.polarity, predicted_polarity)<jupyter_output><empty_output><jupyter_text>TextBlob for Sentiment Analysis<jupyter_code>sample_positive = train.text.loc[256332]
print(sample_positive)
parsed_positive = TextBlob(sample_positive)
parsed_positive.polarity
sample_negative = train.text.loc[636079]
print(sample_negative)
parsed_negative = TextBlob(sample_negative)
parsed_negative.polarity
def estimate_polarity(text):
    return TextBlob(text).sentiment.polarity
train[['text']].sample(10).assign(sentiment=lambda x: x.text.apply(estimate_polarity)).sort_values('sentiment')<jupyter_output><empty_output><jupyter_text>Compare with TextBlob Polarity Score We also obtain TextBlob sentiment scores for the tweets and note (see left panel in below figure) that positive test tweets receive a significantly higher sentiment estimate. We then use the MultinomialNB ‘s model .predict_proba() method to compute predicted probabilities and compare both models using the respective Area Under the Curve (see right panel below).<jupyter_code>test['sentiment'] = test.text.apply(estimate_polarity)
accuracy_score(test.polarity, (test.sentiment>0).astype(int))<jupyter_output><empty_output><jupyter_text>ROC AUC Scores<jupyter_code>roc_auc_score(y_true=test.polarity, y_score=test.sentiment)
roc_auc_score(y_true=test.polarity, y_score=nb.predict_proba(test_dtm)[:, 1])
fpr_tb, tpr_tb, _ = roc_curve(y_true=test.polarity, y_score=test.sentiment)
roc_tb = pd.Series(tpr_tb, index=fpr_tb)
fpr_nb, tpr_nb, _ = roc_curve(y_true=test.polarity, y_score=nb.predict_proba(test_dtm)[:, 1])
roc_nb = pd.Series(tpr_nb, index=fpr_nb)<jupyter_output><empty_output><jupyter_text>The Naive Bayes model outperforms TextBlob in this case.<jupyter_code>fig, axes = plt.subplots(ncols=2, figsize=(14, 6))
sns.boxplot(x='polarity', y='sentiment', data=test, ax=axes[0])
axes[0].set_title('TextBlob Sentiment Scores')
roc_nb.plot(ax=axes[1], label='Naive Bayes', legend=True, lw=1, title='ROC Curves')
roc_tb.plot(ax=axes[1], label='TextBlob', legend=True, lw=1)
sns.despine()
fig.tight_layout();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.108731241473397, 'fraction_numerical': 0.018281036834924966, 'mean_word_length': 4.251432664756447, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 53, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '234132', 'license': 'MIT', 'n_tokens_mistral': 2468, 'n_tokens_neox': 2319, 'n_words': 575, 'path': '14_working_with_text_data/05_sentiment_analysis_twitter.ipynb', 'repo_name': 'driscolljt/Machine-Learning-for-Algorithmic-Trading-Second-Edition_Original', 'size': '7330'}