<jupyter_start><jupyter_text>FAKE NEWS DETECTION PROJECT  Introduction:The authenticity of Information has become a longstanding issue affecting businesses and society, both for printed and digital media. On social networks, the reach and effects of information spread occur at such a fast pace and so amplified that distorted, inaccurate, or false information acquires a tremendous potential to cause real-world impacts, within minutes, for millions of users. Recently, several public concerns about this problem and some approaches to mitigate the problem were expressed. About Dataset:The dataset is in the form of .CSV (Comma Seperated Value) format and consists of **6 columns (5 features and 1 label) with 20800 number of records** as explained below:- `id`: Unique id for each news article.- `headline`: It is the title of the news.- `news`: It contains the full text of the news article.- `Unnamed0`: It is a serial number.- `written_by`: It represent the author of the news article.- `label`: It tells whether the news is fake(1) or not fake(0).<jupyter_code>#importing required libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Load Dataset<jupyter_code>#loading dataset into dataframe
df_fake_news = pd.read_csv("train_news.csv")

#interpreting first 5 rows
df_fake_news.head()<jupyter_output><empty_output><jupyter_text>Data Analysis<jupyter_code>#checking general information of dataset
df_fake_news.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 20800 entries, 0 to 20799
Data columns (total 6 columns):
 #   Column      Non-Null Count  Dtype 
---  ------      --------------  ----- 
 0   Unnamed: 0  20800 non-null  int64 
 1   id          20800 non-null  int64 
 2   headline    20242 non-null  object
 3   written_by  18843 non-null  object
 4   news        20761 non-null  object
 5   label       20800 non-null  int64 
dtypes: int64(3), object(3)
memory usage: 975.1+ KB<jupyter_text>Remarks:- There are **20800 records with 6 columns (3 int type and 3 object type)**.- **Non-Null count shows the presence of null values in the dataset** which needs to be treated accordingly.<jupyter_code>#checking null values
df_fake_news.isnull().sum()
#interpreting null values using heatmap
plt.figure(figsize=(8,6))
sns.heatmap(df_fake_news.isnull())
plt.show()<jupyter_output><empty_output><jupyter_text>Remarks:- From the above interpretion it is clear that the feature `headline`, `written_by` and `news` containes 558, 1957 and 39 null values respectively and needs to be treated accordingly.  Checking feature `Unnamed: 0`<jupyter_code>#interpreting feature
df_fake_news["Unnamed: 0"]<jupyter_output><empty_output><jupyter_text>Remarks:- This feature contains continueos data and represents the serial number of record therefore it can be **dropped** from dataset as it is of no use in model training.  Checking feature `id`<jupyter_code>#interpreting feature
df_fake_news["id"]<jupyter_output><empty_output><jupyter_text>Remarks:- This feature contains continuous data and represents the id of the record, therefore, it can be **dropped** from the dataset as it is of no use in model training.  Checking feature `headline`<jupyter_code>#interpreting feature
df_fake_news["headline"]<jupyter_output><empty_output><jupyter_text>Remarks:- This feature contains **textual data and is of object type** and needs to be handled accordingly.  Checking feature `written_by`<jupyter_code>#interpreting feature
df_fake_news["written_by"]<jupyter_output><empty_output><jupyter_text>Remarks:- This feature contains **textual data and is of object type** and needs to be handled accordingly.  Checking feature `news`<jupyter_code>#interpreting feature
df_fake_news["news"]<jupyter_output><empty_output><jupyter_text>Remarks:- This feature contains **textual data and is of object type** and needs to be handled accordingly.  Checking target `label`<jupyter_code>#interpreting target
df_fake_news["label"]<jupyter_output><empty_output><jupyter_text>Remarks:- The target `label` contains **categorical data which is 0 (not fake) and 1(fake)**.<jupyter_code>#checking the ratio of data between label 0 and 1.
label_0 = len(df_fake_news.label[df_fake_news.label == 0])
label_1 = len(df_fake_news.label[df_fake_news.label == 1])
total_record = len(df_fake_news)

print("Total Records for label[0]: ",label_0)
print("% of Records for label[0] : ",round(label_0*100/total_record,2))
print("\n")
print("Total Records for label[1]: ",label_1)
print("% of Records for label[1] : ",round(label_1*100/total_record,2))<jupyter_output>Total Records for label[0]:  10387
% of Records for label[0] :  49.94


Total Records for label[1]:  10413
% of Records for label[1] :  50.06<jupyter_text>Remarks:- Dataset contains almost equal number of records for label 0 and label 1,therefore, dataset is balanced.  Data Cleansing<jupyter_code>#importing required libraries
import nltk
from nltk.tokenize import regexp_tokenize, word_tokenize
from nltk.corpus import stopwords
from collections import Counter<jupyter_output><empty_output><jupyter_text>Dropping columns `Unnamed: 0` and `id`<jupyter_code>#dropping columns
cols_to_drop = ['Unnamed: 0','id']

#storing into new dataframe object
df = df_fake_news.drop(columns=cols_to_drop)

#interpreting first 5 rows
df.head()<jupyter_output><empty_output><jupyter_text>Treating Null Values: Replacing null values with ' ' (single space) and merging feature `headline`, `written_by` and `news` to new column `text_feature` and dropping columns `headline`, `written_by` and `news`<jupyter_code>#treating null values with ' '
df.fillna(' ',inplace=True)

#Merging feature headline, written_by and news to text_feature
df_news = df.copy()
df_news["text_feature"] = df.headline + " " + df.written_by + " "+ df.news

#dropping feature headline, written_by and news
feature_to_drop = ['headline','written_by','news']
df_news.drop(columns=feature_to_drop,inplace=True)

#interpreting first 5 rows
df_news.head()<jupyter_output><empty_output><jupyter_text>Converting `text_feature` to lower case and storing the length of each news to column original_length<jupyter_code>#converting to lowercase
df_news["text_feature"] = df_news.text_feature.str.lower()

#storing original news length
df_news["original_length"] = df_news.text_feature.str.len()

#interpreting first 5 rows
df_news.head()<jupyter_output><empty_output><jupyter_text>Removing & Replacing certain terms<jupyter_code>#Replacing '\n' with white space (' ')
df_news.text_feature = df_news.text_feature.str.replace('\n',' ')

#Keeping only text and number data i.e., a to z, 0 to 9 and words like hasn't, cann't shouldn't etc.
df_news.text_feature = df_news.text_feature.apply(lambda x: ' '.join(regexp_tokenize(x,"[a-z'0-9]+")))
#replacing numbers i.e., 0 to 9 with numbr
df_news.text_feature = df_news.text_feature.str.replace("\d+","numbr")

#interpreting random 5 rows to see changes
df_news.sample(5)<jupyter_output><ipython-input-18-d78fda722d08>:2: FutureWarning: The default value of regex will change from True to False in a future version.
  df_news.text_feature = df_news.text_feature.str.replace("\d+","numbr")<jupyter_text>Getting Most Common Words<jupyter_code>#instanciating Counter
def get_common_words(texts):
    cnt = Counter()
    for text in texts:
        for word in text.split():
            cnt[word] += 1
            
    return cnt

#getting most common words of text_feature before removal of stopwords
common_words_with_stopwords = get_common_words(df_news.text_feature)

#interpreting most common words
common_words_with_stopwords.most_common(150)<jupyter_output><empty_output><jupyter_text>Remarks:- **In top 150 common words**, *most of the words in the news corpus are stopwords* i.e., the, to, of, and, a, in, that, etc. and therefore, it **needs to be removed.**  Removing Stopwords<jupyter_code>#initialzing stop words as set
stop_words = set(stopwords.words('english'))
print("Original Stopwords: ",len(stop_words))

#adding custom stop words i.e., a to z, to stop_words set
for ch in range(ord('a'),ord('z')+1):
    stop_words.update(chr(ch))

stop_words.update(['could','would','us','also','mr','ms','mrs','among','put'])
print(stop_words)
print("Total Stopwords: ",len(stop_words))
#removing stopwords
df_news.text_feature = df_news.text_feature.apply(lambda x: ' '.join(word for word in x.split() if word not in stop_words))
#interpreting random 5 rows to see the change
df_news.sample(5)<jupyter_output><empty_output><jupyter_text>Removing Punctuations & Storing length of text after cleansing in column new_length<jupyter_code>#removing punctuations
df_news.text_feature = df_news.text_feature.str.replace("[^\w\d\s]","")

#storing length of each text in column new_length
df_news["new_length"] = df_news.text_feature.str.len()

#interpreting random 10 rows to see the change
df_news.sample(10)
#checking total length of text cleaned
original_length_total = df_news.original_length.sum()
new_length_total = df_news.new_length.sum()
length_cleaned = original_length_total - new_length_total

print("Total Original Text Length:",original_length_total)
print("Total New Text Length     :",new_length_total)
print("Length Cleaned:           :",length_cleaned)
print(f"% of Length Cleaned       : {round(length_cleaned*100/original_length_total,2)}%")
#check most common words after removal of stopwords & punctuations
common_words_without_stopwords = get_common_words(df_news.text_feature)

#interpreting top 150 most common words
common_words_without_stopwords.most_common(150)<jupyter_output><empty_output><jupyter_text>Remarks:- After removal of stopwords and punctuations, most common words becomes numbr, said, trump, etc.  Exploratory Data Analysis (EDA)<jupyter_code>#checking target label using count plot
ax = sns.countplot(x="label",data=df_news)
k=0
for i in ax.patches:
    plt.text(k,i.get_height()/2,f"{round(i.get_height()*100/len(df_news.label),2)}%",ha="center",rotation=90,fontweight="bold")
    k += 1
    
ax.set_xticklabels(['0: NotFake','1: Fake'])
plt.title("CountPlot for target label")
plt.show()<jupyter_output><empty_output><jupyter_text>Remarks:- There are almost equal number of records available for fake (1) and Not Fake (0) label.- **Dataset is balanced.**  Checking the news text length distribution before and after cleansing<jupyter_code>#checking the text length distribution using column original_length and new_length
fig, (ax1,ax2) = plt.subplots(1,2,figsize=(15,4))

#for original_length
sns.histplot(df_news.original_length,ax=ax1,color="red",stat="density",kde=True)
ax1.set_title("Original Text Length Distribution")
ax1.set_xlabel("Text Length")

#for new_length
sns.histplot(df_news.new_length,ax=ax2,color="green",stat="density",kde=True)
ax2.set_title("Cleaned Text Length Distribution")
ax2.set_xlabel("Text Length")

plt.show()<jupyter_output><empty_output><jupyter_text>Remarks:- From the above dipiction, it is clear that the **length of text reduced by greater amount** after cleansing of text data.- Also, the range of **text length reduced from 0-140000 to 0-80000**.  Checking the 50 most common words before cleansing and after cleansing<jupyter_code>#creating most common words dataframe
#before cleansing
df_cw_bc = pd.DataFrame(common_words_with_stopwords.most_common(),columns=['words','count'])

#after cleansing
df_cw_ac = pd.DataFrame(common_words_without_stopwords.most_common(),columns=['words','count'])
#Checking the 30 most common words before and after cleansing using barplot
max_words = 30
fig, (ax1,ax2) = plt.subplots(1,2,figsize=(16,8))
fig.subplots_adjust(wspace=0.5)
#before cleansing
sns.barplot(x='count',y='words',data=df_cw_bc[:max_words],ax=ax1)
ax1.set_title("Most Common Words: BEFORE CLEANSING")

#after cleansing
sns.barplot(x='count',y='words',data=df_cw_ac[:max_words],ax=ax2)
ax2.set_title("Most Common Words: AFTER CLEANSING")

plt.show()<jupyter_output><empty_output><jupyter_text>Remarks:- From above dipiction, it is clear that the most common words before cleansing were the, to, of, and, a, etc. while after cleansing most common words are numbr, trump, one, new, people, etc.  WordCloud: Getting sense of loud words<jupyter_code>#importing required libraries
from wordcloud import WordCloud
#getting wordcloud for fake news i.e. label = 1
word_cloud_1 = WordCloud(width=500,height=500,background_color="white",max_words=80).generate(' '.join(df_news.text_feature[df_news.label==1]))

#getting wordcloud for not fake news i.e. label = 0
word_cloud_0 = WordCloud(width=500,height=500,background_color="white",max_words=80).generate(' '.join(df_news.text_feature[df_news.label==0]))

#displaying wordcloud using matplotlib
fig, (ax1,ax2) = plt.subplots(1,2,figsize=(15,6))
fig.subplots_adjust(wspace=0.3)
fig.suptitle("WordCloud: Representation of Loud Words in News Corpus",fontsize=16)

#displaying wordcloud for label = 1
ax1.imshow(word_cloud_1)
ax1.set_title("Loud Words in Fake News")
for spine in ax1.spines.values():
    spine.set_edgecolor("red")

ax1.set_xticks([])
ax1.set_yticks([])
#displaying wordcloud for label = 0
ax2.imshow(word_cloud_0)
ax2.set_title("Loud Words in Not Fake News")
for spine in ax2.spines.values():
    spine.set_edgecolor("green")

ax2.set_xticks([])
ax2.set_yticks([])

fig.tight_layout(pad=2)
plt.show()<jupyter_output><empty_output><jupyter_text>Remarks:- Loud words in Fake news are numbers, hillary clinton, donald trump, america, united state, people, war, etc.- Loud words in Not Fake news are new york, trump, united state, support, help, democrat etc.  EDA Concluding Remarks:>Starting with **count plot**, it was found that dataset is balanced with almost equal number of records for fake news as well as not fake news. Moving further with **distribution plot** for news text length, it was found that after cleansing of news text, the length of news text reduced by a greater amount. Also with the help of **barplot** for most common words, it was found that, before cleansing, the most common words are mostly stop words i.e. the, and, of, a, etc. and after cleansing, the most common words are like, numbr, said, clinton, people, president, time, state etc. With the help of word cloud, it was found that the loud words in fake news are hillary clinton, donald trump, america, united state, war etc. while in not fake news are new york, trump, united state, support, help, demorat etc.  Preparing Data for Model Training & Testing<jupyter_code>#importing required libraries
from sklearn.model_selection import train_test_split, GridSearchCV, cross_val_score
from sklearn.metrics import classification_report, log_loss, accuracy_score
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.linear_model import SGDClassifier, LogisticRegression
from sklearn.experimental import enable_hist_gradient_boosting
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.tree import DecisionTreeClassifier
import tqdm.notebook as tqdm
import sys, timeit
from IPython.display import display<jupyter_output><empty_output><jupyter_text>1. Convert text to vectors<jupyter_code>#convert news text to vectors using TfidfVectorizer
tfidf = TfidfVectorizer(max_features=8000)
features = tfidf.fit_transform(df_news.text_feature).toarray()<jupyter_output><empty_output><jupyter_text>2. Seperating Input and Output Variables<jupyter_code>#Input variable
X = features
print("Feature's Shape: ",X.shape)

#Output variable
Y = df_news.label
print("Target's Shape: ",Y.shape)<jupyter_output>Feature's Shape:  (20800, 8000)
Target's Shape:  (20800,)<jupyter_text>3. Training & Testing Model<jupyter_code>#function to get best random state
def get_best_random_state(model,X,Y,t_size=0.25,rs_range=range(1,301,50)):
    best_rstate = 0
    best_accuracy_score = 0
    random_state_message = "\r"
    
    for i in tqdm.tqdm(rs_range,desc=f"Best_Random_State => {model}"):
        X_train, X_test, Y_train, Y_test = train_test_split(X,Y,test_size=t_size,random_state=i)
        model.fit(X_train, Y_train)
        y_pred = model.predict(X_test)
        a_score = accuracy_score(Y_test,y_pred)
        
        if a_score > best_accuracy_score:
            best_accuracy_score = a_score
            best_rstate = i
            
        random_state_message += f"[{i}: {round(a_score*100,2)}]<--->"
        sys.stdout.write(random_state_message)
        
    sys.stdout.write(f"\n\nBest Random State: {best_rstate} found with Accuracy: {best_accuracy_score}")
    return best_rstate, best_accuracy_score
#End of function


#function to get best cv score
def get_best_cv(model,X_train,Y_train,parameters,cv_range=range(5,25,5)):
    best_cv_score = 0
    best_cv = 0
    
    cv_message = "\r"
    for i in tqdm.tqdm(cv_range,desc=f"Best_CV => {model}"):
        gscv = GridSearchCV(model,parameters)
        gscv.fit(X_train,Y_train)
        
        cv_score = cross_val_score(gscv.best_estimator_,X_train,Y_train,cv=i).mean()
        
        if cv_score > best_cv_score:
            best_cv_score = cv_score
            best_cv = i
        
        cv_message += f"[{i}:{round(cv_score*100,2)}]<--->"
        sys.stdout.write(cv_message)
    
    sys.stdout.write(f"\n\nBest CV: {best_cv} found with Cross Val Score: {best_cv_score}")
    
    return best_cv, best_cv_score
#End of function

#function to build models
def build_models(models,X,Y,t_size=0.25,rs_range=range(1,301,50),cv_range=range(5,25,5)):
    for i in tqdm.tqdm(models,desc="Building Models"):
        sys.stdout.write("\n=======================================================================================\n")
        sys.stdout.write(f"Current Model in Progress: {i} ")
        sys.stdout.write("\n=======================================================================================\n")
        
        #start time
        start_time = timeit.default_timer()
        
        #Find the best random state
        best_random_state, best_accuracy_score = get_best_random_state(models[i]['name'],X,Y,t_size,rs_range)
        sys.stdout.write("\n")
        
        #Spliting train and test data using train_test_split method with best random state value
        X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=t_size,random_state=best_random_state)
        
        #Find the best CV
        best_cv, best_cv_score = get_best_cv(models[i]['name'],X_train,Y_train,models[i]['parameters'],cv_range)
        sys.stdout.write("\n\nBuilding Model...")
        
        #Training the model using best CV
        gscv = GridSearchCV(models[i]['name'],models[i]['parameters'],cv=best_cv)
        gscv.fit(X_train,Y_train)
        
        #Testing model
        y_pred = gscv.best_estimator_.predict(X_test)
        
        #Recording model performance
        model_accuracy_score = accuracy_score(Y_test,y_pred)
        model_classification_report = classification_report(Y_test,y_pred)
        model_log_loss = log_loss(Y_test,y_pred)
        
        #end time
        end_time = timeit.default_timer()
        sys.stdout.write(f"Completed in [{end_time-start_time} sec.]")
        
        #storing model specifications
        models[i]['initial_accuracy_score'] = best_accuracy_score
        models[i]['best_random_state'] = best_random_state
        models[i]['x_train'] = X_train
        models[i]['x_test'] = X_test
        models[i]['y_train'] = Y_train
        models[i]['y_test'] = Y_test
        models[i]['best_cv'] = best_cv
        models[i]['best_cv_score'] = best_cv_score
        models[i]['gscv'] = gscv
        models[i]['y_predict'] = y_pred
        models[i]['final_accuracy'] = model_accuracy_score
        models[i]['log_loss'] = model_log_loss
        models[i]['classification_report'] = model_classification_report
        models[i]['build_time'] = f"{end_time - start_time} (in sec.)"
        
        sys.stdout.write("\n=======================================================================================\n\n\n")
    
    return models
#End of function

#function to display model performance
def display_performance(models):
    model_names = []
    model_initial_score = []
    model_cross_val_score = []
    model_log_loss_score = []
    model_final_score = []
    model_build_time = []
    for i in models:
        model_names.append(i)
        model_initial_score.append(models[i]['initial_accuracy_score'])
        model_cross_val_score.append(models[i]['best_cv_score'])
        model_log_loss_score.append(models[i]['log_loss'])
        model_final_score.append(models[i]['final_accuracy'])
        model_build_time.append(models[i]['build_time'])
    
    model_df = pd.DataFrame({
        "Name": model_names,
        "Initial Score": model_initial_score,
        "Cross Val Score": model_cross_val_score,
        "Log Loss": model_log_loss_score,
        "Final Score": model_final_score,
        "Build Time": model_build_time,
    })
    
    model_df['Difference (Final Score - Cross Val Score)'] = model_df['Final Score'] - model_df['Cross Val Score']
    display(model_df)
    
    for i in models:
        print("========================================================")
        print(f"for model: {i}")
        print("========================================================")
        print("CLASSIFICATION REPORT")
        print(models[i]['classification_report'])
        print("=======================================================\n\n")
        
    return
#End of function
#List of models for training & testing
models = {
    "MultinomialNB":{
        "name": MultinomialNB(),
        "parameters":{
            "alpha": [1.0]                
        }
    },
    "SGDClassifier":{
        "name": SGDClassifier(),
        "parameters":{
            "loss":['hinge','modified_huber'],
            "alpha":[0.001,0.0001,0.00001],
            "n_jobs":[-1],
            "learning_rate":['optimal'],
            "max_iter":[100]
        }
    },
    "HistGradientBoostingClassifier":{
        "name": HistGradientBoostingClassifier(),
        "parameters":{
            "loss": ['binary_crossentropy'],
            "l2_regularization": [0,1.0]
        }
    }
}
#building models
build_model = build_models(models,X,Y,rs_range=[38,40,42,44],cv_range=[5,7,9])
#displaying model performances
display_performance(build_model)<jupyter_output><empty_output><jupyter_text>Remarks:> From the above model comparision it is clear that **HistGradientBoostingClassifier** performs better with **Accuracy Score: 98.03% and Log Loss: 0.67** than other models. Therefore, proceeding with **HistGradientBoostingClassifier**.  Model Saving or Serialization<jupyter_code>import joblib
#selecting best model
best_model = build_model['HistGradientBoostingClassifier']['gscv']

#saving model
joblib.dump(best_model.best_estimator_,open('fake_news_detection.obj','wb'))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12739075502113015, 'fraction_numerical': 0.01738334858188472, 'mean_word_length': 5.0120481927710845, 'pattern_counts': {'":': 23, '<': 113, '<?xml version=': 0, '>': 119, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '28', 'dir': 'jupyter-structured-clean-dedup', 'id': '276546', 'license': 'MIT', 'n_tokens_mistral': 7667, 'n_tokens_neox': 7278, 'n_words': 1911, 'path': 'fake-news-detection/fake-news-detection-project.ipynb', 'repo_name': 'ashok-kumar-sharma/ds-internship', 'size': '22953'}