<jupyter_start><jupyter_text>Ham or Spam<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

%matplotlib inline 

df = pd.read_csv('emails.csv')

df.head()
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 5728 entries, 0 to 5727
Data columns (total 2 columns):
 #   Column  Non-Null Count  Dtype 
---  ------  --------------  ----- 
 0   text    5728 non-null   object
 1   spam    5728 non-null   int64 
dtypes: int64(1), object(1)
memory usage: 89.6+ KB<jupyter_text>Data Preprocessing  Splitting the dataset<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(df.text, df.spam, test_size=0.3, random_state=0)
X_train.shape<jupyter_output><empty_output><jupyter_text>Extracting features from text filesEmails are actually `series of words` (ordered). In order to run machine learning algorithms we need to convert the text of email into `numerical feature vectors`. We have used the `bag of words` model for our example. Briefly, we segment each email into words (for English splitting by space), and **`count number of times each word occurs in each document and finally assign each word an integer id`**. Each unique word in our dictionary will correspond to a feature (descriptive feature).`Scikit-learn` has a high level component `CountVectorizer` which will create feature vectors.<jupyter_code>from sklearn.feature_extraction.text import CountVectorizer

count_vec = CountVectorizer()
X_train_counts = count_vec.fit_transform(X_train)

X_train_counts.shape<jupyter_output><empty_output><jupyter_text>**`TF:`**     Just counting the number of words in each email has 1 issue: it will give more weightage to longer emails than shorter emails. To avoid this, we can use frequency **`(TF - Term Frequencies)`** i.e. **`count(word) / Total words`**, in each email.**`TF-IDF:`** Finally, we have reduce the weightage of more common words like `(the, is, an etc.)` which occurs in all email. This is called as **`TF-IDF i.e Term Frequency times inverse document frequency.`**To achieve this `TfidfTransformer`from `Scikit-learn` is used.<jupyter_code>from sklearn.feature_extraction.text import TfidfTransformer

tfidf_transformer = TfidfTransformer()
X_train_tfidf = tfidf_transformer.fit_transform(X_train_counts)

X_train_tfidf.shape<jupyter_output><empty_output><jupyter_text>Creating the model<jupyter_code>from sklearn.model_selection import cross_val_score, GridSearchCV, StratifiedKFold

skf = StratifiedKFold(n_splits=10)<jupyter_output><empty_output><jupyter_text>Cross Validation<jupyter_code>from sklearn.naive_bayes import MultinomialNB

score = cross_val_score(MultinomialNB(), X_train_tfidf, y_train, cv=skf)
score.mean()<jupyter_output><empty_output><jupyter_text>Parameter Tuning<jupyter_code>parameters = {
    'alpha': (1e-2, 1e-3)
}

gs_nb = GridSearchCV(MultinomialNB(), parameters, cv=skf, n_jobs=-1)
gs_nb = gs_nb.fit(X_train_tfidf, y_train)
print(gs_nb.best_score_)
print(gs_nb.best_params_)<jupyter_output>0.9925174563591022
{'alpha': 0.01}<jupyter_text>Pipeline<jupyter_code>from sklearn.pipeline import Pipeline
model_nb = Pipeline([
                ('vect', CountVectorizer()),
                ('tfidf', TfidfTransformer()),
                ('clf', MultinomialNB(alpha=0.01))
            ])
# Parameter Tuning

parameters = {
    'vect__ngram_range': [(1, 1), (1, 2)],
    'tfidf__use_idf': (True, False),
    'clf__alpha': (1e-2, 1e-3)
}

gs_nb = GridSearchCV(model_nb, parameters, n_jobs=-1)
gs_nb = gs_nb.fit(X_train, y_train)
print(gs_nb.best_score_)
print(gs_nb.best_params_)<jupyter_output>0.991021198564139
{'clf__alpha': 0.01, 'tfidf__use_idf': True, 'vect__ngram_range': (1, 2)}<jupyter_text>Fitting the model<jupyter_code>gs_nb.fit(X_train, y_train);
# Semi-colon at the end to not to print the output<jupyter_output><empty_output><jupyter_text>Prediction<jupyter_code>y_test_pred = gs_nb.predict(X_test)<jupyter_output><empty_output><jupyter_text>Evaluating the model<jupyter_code>from sklearn.metrics import f1_score, precision_score, recall_score
# Metrics

print(f'Model Score: {gs_nb.score(X_test, y_test)}')
print(f'f1-score: {f1_score(y_test, y_test_pred, average="weighted")}')
print(f'precision score: {precision_score(y_test, y_test_pred, average="weighted")}')
print(f'recall score: {recall_score(y_test, y_test_pred, average="weighted")}')
from sklearn.metrics import classification_report, confusion_matrix
import itertools

def plot_confusion_matrix(cm, classes,
                          normalize=False,
                          title='Confusion matrix',
                          cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    Normalization can be applied by setting `normalize=True`.
    """
    if normalize:
        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
        print("Normalized confusion matrix")
    else:
        print('Confusion matrix, without normalization')

    print(cm)

    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)

    fmt = '.2f' if normalize else 'd'
    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, format(cm[i, j], fmt),
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")

    plt.tight_layout()
    plt.ylabel('True label')
    plt.xlabel('Predicted label')
cnf_matrix = confusion_matrix(y_test, y_test_pred, labels=[0,1])
np.set_printoptions(precision=2)

print(classification_report(y_test, y_test_pred))

# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=['Ham(0)', 'Spam(1)'],normalize= False,  title='Confusion matrix')<jupyter_output>precision    recall  f1-score   support

           0       0.99      1.00      0.99      1314
           1       1.00      0.97      0.98       405

    accuracy                           0.99      1719
   macro avg       0.99      0.98      0.99      1719
weighted avg       0.99      0.99      0.99      1719

Confusion matrix, without normalization
[[1313    1]
 [  14  391]]<jupyter_text>Saving the model<jupyter_code>from joblib import dump

dump(gs_nb, 'model.joblib')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11456400742115028, 'fraction_numerical': 0.027674706246134818, 'mean_word_length': 4.285130718954249, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '542031', 'license': 'MIT', 'n_tokens_mistral': 2356, 'n_tokens_neox': 2220, 'n_words': 524, 'path': 'venv/src/notebook.ipynb', 'repo_name': 'AkashSDas/Ham-or-Spam', 'size': '6468'}