<jupyter_start><jupyter_text>Sentiment Analysis IntroductionThis notebook contains sentiment analysis for the Batman movie screenplays. A few key points to remember with sentiment analysis.1. **TextBlob Module**: Linguistic researchers have labeled the sentiment of words based on their domain expertise. Sentiment of words can vary based on where it is in a sentence. The TextBlob module allows us to take advantage of these labels.2. **Sentiment Labels**: Each word in a corpus is labeled in terms of polarity and subjectivity (there are more labels as well, but we're going to ignore them for now). A corpus' sentiment is the average of these.* **Polarity**: How positive or negative a word is. -1 is very negative. +1 is very positive.* **Subjectivity**: How subjective, or opinionated a word is. 0 is fact. +1 is very much an opinion.Let's take a look at the sentiment of the various scene dialogues.<jupyter_code># data manipulation
import pandas as pd
import numpy as np
import math

# sentiment analysis
from textblob import TextBlob

# visualization
import matplotlib.pyplot as plt
# read in the corpus
df = pd.read_pickle('df.pkl')
# create functions to find the polarity and subjectivity of each routine
pol = lambda x: TextBlob(x).sentiment.polarity
sub = lambda x: TextBlob(x).sentiment.subjectivity

df['polarity'] = df['dialogue_lemmatized'].apply(pol)
df['subjectivity'] = df['dialogue_lemmatized'].apply(sub)
df
# plot the results

plt.rcParams['figure.figsize'] = [10, 8]

for index, scene in enumerate(df.index):
    x = df.polarity.loc[scene]
    y = df.subjectivity.loc[scene]
    plt.scatter(x, y, color='blue')
    #plt.text(x+.001, y+.001, df['dialogue'][index], fontsize=10)
    plt.xlim(-.01, .12) 
    
plt.title('Sentiment Analysis', fontsize=20)
plt.xlabel('<-- Negative -------- Positive -->', fontsize=15)
plt.ylabel('<-- Facts -------- Opinions -->', fontsize=15)

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11254517294785751, 'fraction_numerical': 0.013939081053175014, 'mean_word_length': 4.568965517241379, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '372126', 'license': 'MIT', 'n_tokens_mistral': 596, 'n_tokens_neox': 556, 'n_words': 230, 'path': 'batman_sentiment_analysis.ipynb', 'repo_name': 'wnobles/metis-finalproject-batmanthemes', 'size': '1937'}