<jupyter_start><jupyter_text>Fake news Detection ![I-Newspaper2.jpg](attachment:I-Newspaper2.jpg)  Importing required libraryHere I am going to importing some of the required library, if extra library is required to install It will be install later on.<jupyter_code>import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from sklearn.metrics import classification_report
import re
import string<jupyter_output><empty_output><jupyter_text>Inserting fake and real dataset<jupyter_code>df_fake = pd.read_csv("Fake.csv")
df_true = pd.read_csv("True.csv")
df_fake.head(5)
df_true.head(5)<jupyter_output><empty_output><jupyter_text>Inserting a column called "class" for fake and real news dataset to categories fake and true news.<jupyter_code>df_fake["class"] = 0
df_true["class"] = 1<jupyter_output><empty_output><jupyter_text>Removing last 10 rows from both the dataset, for manual testing<jupyter_code>df_fake.shape, df_true.shape
df_fake_manual_testing = df_fake.tail(10)
for i in range(23480,23470,-1):
    df_fake.drop([i], axis = 0, inplace = True)
df_true_manual_testing = df_true.tail(10)
for i in range(21416,21406,-1):
    df_true.drop([i], axis = 0, inplace = True)
df_fake.shape, df_true.shape<jupyter_output><empty_output><jupyter_text>Merging the manual testing dataframe in single dataset and save it in a csv file<jupyter_code>df_fake_manual_testing["class"] = 0
df_true_manual_testing["class"] = 1
df_fake_manual_testing.head(10)
df_true_manual_testing.head(10)
df_manual_testing = pd.concat([df_fake_manual_testing,df_true_manual_testing], axis = 0)
df_manual_testing.to_csv("manual_testing.csv")<jupyter_output><empty_output><jupyter_text>Merging the main fake and true dataframe<jupyter_code>df_marge = pd.concat([df_fake, df_true], axis =0 )
df_marge.head(10)
df_marge.columns<jupyter_output><empty_output><jupyter_text>"title",  "subject" and "date" columns is not required for detecting the fake news, so I am going to drop the columns.<jupyter_code>df = df_marge.drop(["title", "subject","date"], axis = 1)
df.isnull().sum()<jupyter_output><empty_output><jupyter_text>Randomly shuffling the dataframe<jupyter_code>df = df.sample(frac = 1)
df.head()
df.reset_index(inplace = True)
df.drop(["index"], axis = 1, inplace = True)
df.columns
df.head()<jupyter_output><empty_output><jupyter_text>Creating a function to convert the text in lowercase, remove the extra space, special chr., ulr and links.<jupyter_code>def wordopt(text):
    text = text.lower()
    text = re.sub('\[.*?\]', '', text)
    text = re.sub("\\W"," ",text) 
    text = re.sub('https?://\S+|www\.\S+', '', text)
    text = re.sub('<.*?>+', '', text)
    text = re.sub('[%s]' % re.escape(string.punctuation), '', text)
    text = re.sub('\n', '', text)
    text = re.sub('\w*\d\w*', '', text)    
    return text
df["text"] = df["text"].apply(wordopt)<jupyter_output><empty_output><jupyter_text>Defining dependent and independent variable as x and y<jupyter_code>x = df["text"]
y = df["class"]<jupyter_output><empty_output><jupyter_text>Splitting the dataset into training set and testing set.<jupyter_code>x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25)<jupyter_output><empty_output><jupyter_text>Convert text to vectors<jupyter_code>from sklearn.feature_extraction.text import TfidfVectorizer
vectorization = TfidfVectorizer()
xv_train = vectorization.fit_transform(x_train)
xv_test = vectorization.transform(x_test)<jupyter_output><empty_output><jupyter_text>1. Logistic Regression<jupyter_code>from sklearn.linear_model import LogisticRegression
LR = LogisticRegression()
LR.fit(xv_train,y_train)
pred_lr=LR.predict(xv_test)
LR.score(xv_test, y_test)
print(classification_report(y_test, pred_lr))<jupyter_output>precision    recall  f1-score   support

           0       0.99      0.99      0.99      5883
           1       0.99      0.99      0.99      5337

    accuracy                           0.99     11220
   macro avg       0.99      0.99      0.99     11220
weighted avg       0.99      0.99      0.99     11220<jupyter_text>2. Decision Tree Classification<jupyter_code>from sklearn.tree import DecisionTreeClassifier
DT = DecisionTreeClassifier()
DT.fit(xv_train, y_train)
pred_dt = DT.predict(xv_test)
DT.score(xv_test, y_test)
print(classification_report(y_test, pred_dt))<jupyter_output>precision    recall  f1-score   support

           0       1.00      0.99      0.99      5883
           1       0.99      0.99      0.99      5337

    accuracy                           0.99     11220
   macro avg       0.99      0.99      0.99     11220
weighted avg       0.99      0.99      0.99     11220<jupyter_text>3. Gradient Boosting Classifier<jupyter_code>from sklearn.ensemble import GradientBoostingClassifier
GBC = GradientBoostingClassifier(random_state=0)
GBC.fit(xv_train, y_train)
pred_gbc = GBC.predict(xv_test)
GBC.score(xv_test, y_test)
print(classification_report(y_test, pred_gbc))<jupyter_output>precision    recall  f1-score   support

           0       1.00      0.99      1.00      5883
           1       0.99      1.00      1.00      5337

    accuracy                           1.00     11220
   macro avg       1.00      1.00      1.00     11220
weighted avg       1.00      1.00      1.00     11220<jupyter_text>4. Random Forest Classifier<jupyter_code>from sklearn.ensemble import RandomForestClassifier
RFC = RandomForestClassifier(random_state=0)
RFC.fit(xv_train, y_train)
pred_rfc = RFC.predict(xv_test)
RFC.score(xv_test, y_test)
print(classification_report(y_test, pred_rfc))<jupyter_output>precision    recall  f1-score   support

           0       0.99      0.99      0.99      5883
           1       0.99      0.99      0.99      5337

    accuracy                           0.99     11220
   macro avg       0.99      0.99      0.99     11220
weighted avg       0.99      0.99      0.99     11220<jupyter_text>Model Testing With Manual Entry News<jupyter_code>def output_lable(n):
    if n == 0:
        return "Fake News"
    elif n == 1:
        return "Not A Fake News"
    
def manual_testing(news):
    testing_news = {"text":[news]}
    new_def_test = pd.DataFrame(testing_news)
    new_def_test["text"] = new_def_test["text"].apply(wordopt) 
    new_x_test = new_def_test["text"]
    new_xv_test = vectorization.transform(new_x_test)
    pred_LR = LR.predict(new_xv_test)
    pred_DT = DT.predict(new_xv_test)
    pred_GBC = GBC.predict(new_xv_test)
    pred_RFC = RFC.predict(new_xv_test)

    return print("\n\nLR Prediction: {} \nDT Prediction: {} \nGBC Prediction: {} \nRFC Prediction: {}".format(output_lable(pred_LR[0]), 
                                                                                                              output_lable(pred_DT[0]), 
                                                                                                              output_lable(pred_GBC[0]), 
                                                                                                              output_lable(pred_RFC[0])))
news = str(input())
manual_testing(news)<jupyter_output><NAME>In the last years of the 20th century fraud entered US foreign policy in a new way.  On false pretenses Washington dismantled Yugoslavia and Serbia in order to advance an undeclared agenda. In the 21st century this fraud multiplied many times. Afghanistan, Iraq, Somalia, and Libya were destroyed, and Iran and Syria would also have been destroyed if the President of Russia had not prevented it.  Washington is also behind the current destruction of Yemen, and Washington has enabled and financed the Israeli destruction of Palestine.  Additionally, Washington operated militarily within Pakistan without declaring war, murdering many women, children, and village elders under the guise of  combating terrorism.  Washington s war crimes rival those of any country in history.I have documented these crimes in my columns and books (Clarity Press). Anyone who still believes in the purity of Washington s foreign policy is a lost soul  Russia and China now have a strategic alliance [...]
{'fraction_non_alphanumeric': 0.09501158677887547, 'fraction_numerical': 0.040004878643737043, 'mean_word_length': 3.164550533265617, 'pattern_counts': {'":': 1, '<': 66, '<?xml version=': 0, '>': 66, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 23, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '331796', 'license': 'MIT', 'n_tokens_mistral': 2872, 'n_tokens_neox': 2656, 'n_words': 665, 'path': 'Fake_news_detection.ipynb', 'repo_name': 'nikhilwanshi/Fake-News-Detection', 'size': '8211'}