<jupyter_start><jupyter_text>Data Analysis -- Measuring Coronavirus Word Choice and Messaging Focus -- Logistic Regression  How Important Are Specific Word/Vocabulary Differences between Fox and CNN news broadcasts?  From Keyness Analysis, it appears that there are many words that differ significantly between the news outlets' news broadcasts containing coronavirus coverage; there are many words that CNN uses significantly more than Fox does and vice versa in each of their news broadcasts containing coronavirus coverage. It also appears that these words are important for shaping each news outlet's overall messaging and news broadcast content which are key elements of overall news outlet response.  How meaningful are these vocabulary differences? Are these vocabulary differences strong enough to truly distinguish Fox and CNN news coverage about the coronavirus from one another?  In order to answer this question I ran a logistic regression. First, I created a new dataset, taking all 463 Fox news broadcasts and taking a random sample of 463 news broadcasts from the full CNN news broadcast corpus. I then prepared the data and created a document term matrix with the most common 1000 words found between the news broadcasts. I next added a new column to the document term matrix to represent the identity of the news broadcast represented in each row (1 for CNN and 0 for Fox). With this data, I then ran a logistic regression using the 250 most common words found between the news broadcasts as explanatory variables (this was the maximum number of predictors I could use in statsmodels' logit function on this server) and the news outlet identity as the response variable.  In order to run a logistic regression I took the following steps. I first created and analyzed a logit model on the full dataset described above with 250 predictors, in order to find the most important words that distinguish one news outlet's coronavirus coverage from the other's. I categorized the "most important words" as those that ultimately had a coefficient in the resulting model with a p-value of less than 0.01, which would indicate that that word is a significant predictor of news outlet identity at the 99% confidence level. I then set these words aside, and re-ran another logit model just using this narrowed downn set of words. In this second model, I split my data up into training and test datasets, using a 70-30 split, trained the model using the training data, and then tested the resulting model on the test data. I finally analyzed the resulting predictions on the test data and compared it with the training data output in order to assess model precision and accuracy.     If the logit model as a whole is significant and certain words are significant predictors of news outlet identity, it will reveal that vocabulary differences between the news outlets are meaningful and important in distinguishing between their coronavirus responses. This will also show that vocabulary differences are strong enough alone to actually distinguish between Fox and CNN news coverage about the coronavirus from one another. The outcome of this logistic regression will reveal which words (if any) are the most important in distinguishing one news outlet's coverage of coronavirus versus the other's.  It will also confirm the words identified by Keyness Analysis are indeed significanlty different betweenn the news outlets.  Data Prep For Logit Model<jupyter_code>%run data_processing.ipynb
#Take a random subset of the CNN data equal to the Fox data length
random_subset_cnn=data_cnn.sample(n=len(data_fox))
random_subset_cnn2=random_subset_cnn.drop(['index'],axis=1)
#Prep Fox Data As Well
fox_for_pred=data_fox.drop(['index'],axis=1)
#Combine Fox and CNN Data
combined_data=pd.concat([random_subset_cnn2,fox_for_pred],ignore_index=True,sort=False)
word_dist = Counter()

for text in combined_data['targeted text']:
    tokens = tokenize(text,True,strip_chars=strip_chars)
    word_dist.update(tokens)
word_dist_sorted=sorted(word_dist, key=word_dist.get, reverse=True)
vocab = word_dist_sorted[0:1000]
rows=[]
for text in combined_data['targeted text']:
    row = []
    tokens = tokenize(text,True,strip_chars=strip_chars)
    for item in vocab:
        row.append(item in tokens)

    rows.append(row)
dtm=pd.DataFrame(rows,columns=vocab)
news_outlet=[]
for i in combined_data['News Outlet']:
    if i=="CNN":
        news_outlet.append(True)
        
    else:
        news_outlet.append(False)
    
dtm["News Outlet"]=news_outlet
dtm=dtm.astype(int)<jupyter_output><empty_output><jupyter_text>Creating and analyzing the first Logit model on the entire dataset to identify the most important predictors<jupyter_code>cols_x=dtm.columns[0:250]
col_y=dtm.columns[1000]
X=dtm[cols_x]
y=dtm[col_y]
import statsmodels.api as sm
import statsmodels.formula.api as smf
logit_model=sm.Logit(y,X)
result=logit_model.fit()
results_summary=result.summary2()
results_summary<jupyter_output><empty_output><jupyter_text>As shown in the results summary above, the model appears to be highly significant (LLR p-value:	2.8176-79), and there are many words that are significant predictors of a news outlet's identity given all of the other words in the model at both the 0.05 and 0.01 significance levels. The model also has a Pseudo R-squared of 0.725 which is rather surprisingly high.   Select explanatory variables (words) that distinguish between Fox and CNN news transcripts the most. Pick only those words with a p-value equal to or less than 0.01<jupyter_code>results_table=results_summary.tables[1]
most_important_distinguishing_words=results_table[results_table['P>|z|']<0.01].sort_values(['P>|z|'])
most_important_distinguishing_words[0:25]<jupyter_output><empty_output><jupyter_text>There are 36 words that distinguish between CNN and Fox news transcripts at the 99% significance level. They are found in the table above  The most important words (categorized at the 0.01 significance level) found between Fox and CNN news broadcasts about the coronavirus that distinguish between the two can be found above. The most meaningful of these words, excluding filler words, include 'outbreak' , 'covid19', 'symptoms', 'president', and 'hospitals' to name a few. These words are the most important and statistically significant words/predictors that are able to distinguish between Fox and CNN news coverage of the coronavirus. The words above will be used to create a more robust logistic regression model using cross-validation.  Using Sklearn to Predict News Outlet Identity  Creating Second Logistic Regression Model Fitting on training data-- Predictors Narrowed Down<jupyter_code>import sklearn
from sklearn.linear_model import LogisticRegression
from sklearn import metrics
from sklearn.model_selection import train_test_split
#Use only most important words found in the logistic regression above:
cols_x_forPred=most_important_distinguishing_words.index
col_y_forPred=dtm.columns[1000]
X_forPred=dtm[cols_x_forPred]
y_forPred=dtm[col_y_forPred]

X_train, X_test, y_train, y_test = train_test_split(X_forPred, y_forPred, test_size=0.3, random_state=0)
logreg = LogisticRegression()
logreg.fit(X_train, y_train)<jupyter_output>/usr/local/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.
  FutureWarning)<jupyter_text>Predicting the test set results using the above model and calculating the accuracy<jupyter_code>y_pred = logreg.predict(X_test)<jupyter_output><empty_output><jupyter_text>Evaluating the Model  Accuracy<jupyter_code>print('Accuracy of logistic regression classifier on test set: {:.2f}'.format(logreg.score(X_test, y_test)))<jupyter_output>Accuracy of logistic regression classifier on test set: 0.75<jupyter_text>Confusion Matrix<jupyter_code>from sklearn.metrics import classification_report
print('Accuracy of logistic regression classifier on test set: {:.2f}'.format(logreg.score(X_test, y_test)))
print("\n")
print(classification_report(y_test,y_pred))<jupyter_output>Accuracy of logistic regression classifier on test set: 0.75


              precision    recall  f1-score   support

           0       0.75      0.72      0.73       134
           1       0.75      0.77      0.76       144

   micro avg       0.75      0.75      0.75       278
   macro avg       0.75      0.75      0.75       278
weighted avg       0.75      0.75      0.75       278
{'fraction_non_alphanumeric': 0.05556860902255639, 'fraction_numerical': 0.019736842105263157, 'mean_word_length': 5.041873669268985, 'pattern_counts': {'":': 1, '<': 27, '<?xml version=': 0, '>': 28, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '388591', 'license': 'MIT', 'n_tokens_mistral': 2442, 'n_tokens_neox': 2226, 'n_words': 1023, 'path': 'previous_final_project_examples/jdlish/Data Analysis -- Logistic Regression.ipynb', 'repo_name': 'mbod/comm313_spring2021', 'size': '8512'}