<jupyter_start><jupyter_text>Goal of this notebook to test several classifiers on the data set with different features  And beforehand i want to thank <NAME> for his magnificent "Python for Data Science and Machine Learning" course on Udemy , which helped me to dive into ML =)  Let's begin First of all neccesary imports<jupyter_code>import numpy as np
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
import seaborn as sns
import string
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.model_selection import train_test_split
from nltk.stem import SnowballStemmer
from nltk.corpus import stopwords
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Let's read the data from csv file<jupyter_code>sms = pd.read_csv('../input/spam.csv', encoding='latin-1')
sms.head()<jupyter_output><empty_output><jupyter_text>Now drop "unnamed" columns and rename v1 and v2 to "label" and "message"<jupyter_code>sms = sms.drop(['Unnamed: 2','Unnamed: 3','Unnamed: 4'],axis=1)
sms = sms.rename(columns = {'v1':'label','v2':'message'})<jupyter_output><empty_output><jupyter_text>Let's look into our data<jupyter_code>sms.groupby('label').describe()<jupyter_output><empty_output><jupyter_text>Intresting that "Sorry, I'll call later" appears only 30 times here =) Now let's create new feature "message length" and plot it to see if it's of any interest<jupyter_code>sms['length'] = sms['message'].apply(len)
sms.head()
mpl.rcParams['patch.force_edgecolor'] = True
plt.style.use('seaborn-bright')
sms.hist(column='length', by='label', bins=50,figsize=(11,5))<jupyter_output><empty_output><jupyter_text>Looks like the lengthy is the message, more likely it is a spam. Let's not forget this  Text processing and vectorizing our meddages Let's create new data frame. We'll need a copy later on<jupyter_code>text_feat = sms['message'].copy()<jupyter_output><empty_output><jupyter_text>Now define our tex precessing function. It will remove any punctuation and stopwords aswell.<jupyter_code>def text_process(text):
    
    text = text.translate(str.maketrans('', '', string.punctuation))
    text = [word for word in text.split() if word.lower() not in stopwords.words('english')]
    
    return " ".join(text)
text_feat = text_feat.apply(text_process)
vectorizer = TfidfVectorizer("english")
features = vectorizer.fit_transform(text_feat)<jupyter_output><empty_output><jupyter_text>Classifiers and predictions First of all let's split our features to test and train set<jupyter_code>features_train, features_test, labels_train, labels_test = train_test_split(features, sms['label'], test_size=0.3, random_state=111)<jupyter_output><empty_output><jupyter_text>Now let's import bunch of classifiers, initialize them and make a dictionary to itereate through<jupyter_code>from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.naive_bayes import MultinomialNB
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import AdaBoostClassifier
from sklearn.ensemble import BaggingClassifier
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.metrics import accuracy_score
svc = SVC(kernel='sigmoid', gamma=1.0)
knc = KNeighborsClassifier(n_neighbors=49)
mnb = MultinomialNB(alpha=0.2)
dtc = DecisionTreeClassifier(min_samples_split=7, random_state=111)
lrc = LogisticRegression(solver='liblinear', penalty='l1')
rfc = RandomForestClassifier(n_estimators=31, random_state=111)
abc = AdaBoostClassifier(n_estimators=62, random_state=111)
bc = BaggingClassifier(n_estimators=9, random_state=111)
etc = ExtraTreesClassifier(n_estimators=9, random_state=111)<jupyter_output><empty_output><jupyter_text>Parametres are based on notebook:[Spam detection Classifiers hyperparameter tuning][1]  [1]: https://www.kaggle.com/muzzzdy/d/uciml/sms-spam-collection-dataset/spam-detection-classifiers-hyperparameter-tuning/<jupyter_code>clfs = {'SVC' : svc,'KN' : knc, 'NB': mnb, 'DT': dtc, 'LR': lrc, 'RF': rfc, 'AdaBoost': abc, 'BgC': bc, 'ETC': etc}<jupyter_output><empty_output><jupyter_text>Let's make functions to fit our classifiers and make predictions<jupyter_code>def train_classifier(clf, feature_train, labels_train):    
    clf.fit(feature_train, labels_train)
def predict_labels(clf, features):
    return (clf.predict(features))<jupyter_output><empty_output><jupyter_text>Now iterate through classifiers and save the results<jupyter_code>pred_scores = []
for k,v in clfs.items():
    train_classifier(v, features_train, labels_train)
    pred = predict_labels(v,features_test)
    pred_scores.append((k, [accuracy_score(labels_test,pred)]))
df = pd.DataFrame.from_items(pred_scores,orient='index', columns=['Score'])
df
df.plot(kind='bar', ylim=(0.9,1.0), figsize=(11,6), align='center', colormap="Accent")
plt.xticks(np.arange(9), df.index)
plt.ylabel('Accuracy Score')
plt.title('Distribution by Classifier')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)<jupyter_output><empty_output><jupyter_text>Looks like ensemble classifiers are not doing as good as expected.  Stemmer It is said that stemming short messages does no goot or even harm predictions. Let's try this out. Define our stemmer function<jupyter_code>def stemmer (text):
    text = text.split()
    words = ""
    for i in text:
            stemmer = SnowballStemmer("english")
            words += (stemmer.stem(i))+" "
    return words<jupyter_output><empty_output><jupyter_text>Stem, split, fit - repeat... Predict!<jupyter_code>text_feat = text_feat.apply(stemmer)
features = vectorizer.fit_transform(text_feat)
features_train, features_test, labels_train, labels_test = train_test_split(features, sms['label'], test_size=0.3, random_state=111)
pred_scores = []
for k,v in clfs.items():
    train_classifier(v, features_train, labels_train)
    pred = predict_labels(v,features_test)
    pred_scores.append((k, [accuracy_score(labels_test,pred)]))
df2 = pd.DataFrame.from_items(pred_scores,orient='index', columns=['Score2'])
df = pd.concat([df,df2],axis=1)
df
df.plot(kind='bar', ylim=(0.85,1.0), figsize=(11,6), align='center', colormap="Accent")
plt.xticks(np.arange(9), df.index)
plt.ylabel('Accuracy Score')
plt.title('Distribution by Classifier')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)<jupyter_output><empty_output><jupyter_text>Looks like mostly the same . Ensemble classifiers doing a little bit better, NB still got the lead.  What have we forgotten? Message length! Let's append our message length feature to the matrix we fit into our classifiers<jupyter_code>lf = sms['length'].as_matrix()
newfeat = np.hstack((features.todense(),lf[:, None]))
features_train, features_test, labels_train, labels_test = train_test_split(newfeat, sms['label'], test_size=0.3, random_state=111)
pred_scores = []
for k,v in clfs.items():
    train_classifier(v, features_train, labels_train)
    pred = predict_labels(v,features_test)
    pred_scores.append((k, [accuracy_score(labels_test,pred)]))
df3 = pd.DataFrame.from_items(pred_scores,orient='index', columns=['Score3'])
df = pd.concat([df,df3],axis=1)
df
df.plot(kind='bar', ylim=(0.85,1.0), figsize=(11,6), align='center', colormap="Accent")
plt.xticks(np.arange(9), df.index)
plt.ylabel('Accuracy Score')
plt.title('Distribution by Classifier')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)<jupyter_output><empty_output><jupyter_text>This time everyone are doing a little bit worse, except for LinearRegression and RandomForest. But the winner is still MultinominalNaiveBayes.  Voting classifier We are using ensemble algorithms here, but what about ensemble of ensembles? Will it beat NB?<jupyter_code>from sklearn.ensemble import VotingClassifier
eclf = VotingClassifier(estimators=[('BgC', bc), ('ETC', etc), ('RF', rfc), ('Ada', abc)], voting='soft')
eclf.fit(features_train,labels_train)
pred = eclf.predict(features_test)
print(accuracy_score(labels_test,pred))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12147102526002972, 'fraction_numerical': 0.01411589895988113, 'mean_word_length': 7.413541666666666, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 66, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '2924', 'license': 'MIT', 'n_tokens_mistral': 2874, 'n_tokens_neox': 2721, 'n_words': 641, 'path': 'sms-spam-detection-with-various-classifiers.ipynb', 'repo_name': 'pruthiviraj71/Spam-and-Fraudulent-Activites-Detection', 'size': '8083'}