<jupyter_start><jupyter_text>Chapter 01: Breast Cancer Detection**Machine learning**, a subset of artificial intelligence (AI), has taken the world by storm. Within the healthcare domain, it is possible to see how machine learning can make manual processes easier, providing benefits for patients, providers, and pharmaceutical companies alike. Google, for example, has developed a machine learning algorithm that can identify cancerous tumors on mammograms. Similarly, Stanford University has developed a deep learning algorithm to identify skin cancer. Import Necessary ModulesHere, I'll import the necessary modules for this project. I'll group cells together by similar functions.I'll use ``numpy`` and ``scipy`` for array analysis.<jupyter_code># Array analysis
import scipy
import numpy as np<jupyter_output><empty_output><jupyter_text>I'll use ``matplotlib`` and ``pyplot`` to visualize the data and results.<jupyter_code># Data visualize
import matplotlib
from matplotlib import pyplot as plt<jupyter_output><empty_output><jupyter_text>I'll use the ``pandas`` module to both import the data and plot the results along a matrix.<jupyter_code># Data importing and data processing
import pandas as pd
from pandas.plotting import scatter_matrix<jupyter_output><empty_output><jupyter_text>Now I'll import various ``scikit-learn`` modules, classes, functions, etc. to parse the data for training. I'm importing multiple different models to compare how well they predict breast cancers based on the pathologic features I'll be looking at.<jupyter_code># scikit-learn module
from sklearn import preprocessing
from sklearn import model_selection
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.metrics import classification_report
from sklearn.metrics import accuracy_score<jupyter_output><empty_output><jupyter_text>Importing the Breast Cancer Data as a ``DataFrame``The URL below will allow me to access the breast cancer database.<jupyter_code># Load the dataset
url = "https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.data"
"""
str: The link to the dataset.
"""<jupyter_output><empty_output><jupyter_text>The ``list`` below will contain the ``str`` headings for the columns I want. These columns include the following:* ID* Clump thickness* Uniform cell size* Marginal adhesion* Single epithelial size* Bare nuclei* Bland chromatin* Normal nucleoli* Mitoses* Class<jupyter_code># List the columns I want to import
names = [
    "id",
    "clump_thickness",
    "uniform_cell_size",
    "uniform_cell_shape",
    "marginal_adhesion",
    "single_epithelial_size",
    "bare_nuclei",
    "bland_chromatin",
    "normal_nucleoli",
    "mitoses",
    "class"
]
"""
list[str]: The column heading I want to get.
"""
# In the future, I might want to drop the ""id"" column from the list<jupyter_output><empty_output><jupyter_text>Now we'll create the ``DataFrame`` for the breast cancer dataset.<jupyter_code># Create the initial DataFrame
df = pd.read_csv(
    url,
    names = names
)
"""
pandas.core.frame.DataFrame: The DataFrame with my data.
"""
df<jupyter_output><empty_output><jupyter_text>Cleaning the ``DataFrame``Some of the entries are missing and need to be corrected. The missing data is labeled as ``?`` in the ``DataFrame``. Here, I'll change ``?`` into ``-99999``.<jupyter_code># Preprocess the data
find_what = "?"
"""
str: The old entry for missing data.
"""
replace_with = -99999
"""
int: The new entry for missing data.
"""

# Replacing unclear entries with a missing entry marker
df.replace(
    find_what,
    replace_with,
    inplace=True
)<jupyter_output><empty_output><jupyter_text>Now I'll drop the ``id`` column, since I don't need it for training the models.<jupyter_code># Drop the ID column
column_to_drop = ['id']
"""
list[str]: The first column I need to drop.

Here, is the ID for the data.
"""
df.drop(
    column_to_drop,
    1,
    inplace=True
)
df<jupyter_output><empty_output><jupyter_text>Describing and Visualizing the ``DataFrame``Lets visualize one of the rows in the ``DataFrame``.<jupyter_code># Lets explore the dataset and do a few visualizations
row_to_print = 3
"""
int: The row to print in this example.
"""
print(df.shape)
df.loc[row_to_print]<jupyter_output>(699, 10)<jupyter_text>I'll use the ``describe()`` method to get a general overview of what the data tells me.<jupyter_code># Describe the dataset
df.describe()<jupyter_output><empty_output><jupyter_text>Although the ``describe()`` method presents some very interesting data, it isn't the easiest thing to read. That's why I'll plot the data on several histograms to easily see what sticks out to me.<jupyter_code># Plot histograms for each variable.
my_figsize = (10, 10)
"""
tuple[int, int]: The size of the histograms I want to print.
"""
df.hist(figsize = my_figsize)
plt.show()<jupyter_output><empty_output><jupyter_text>I'll now look at a scatterplot matrix, to see the relationships between some of these variables.A scatterplot matrix is a very useful function to use, because it can tell me whether a linear classifier will be a good classifier for my data, or whether I have to investigate more complicated methods.<jupyter_code># Create a scatter plot matrix
my_figsize = (18, 18)
"""
tuple[int, int]: The size of the Pandas scatter plot matrix.

The units are in inches.
"""
scatter_matrix(
    df,
    figsize=my_figsize,
    diagonal='hist'
)
plt.show()<jupyter_output><empty_output><jupyter_text>I can see that there is a pretty strong linear relationship between ``uniform_cell_shape`` and ``uniform_cell_size``. This is expected.When we go through the scatterplot matrix, I can see that some other cells have a good linear relationship. If I look at my classifications, however, there's no easy way to classify these relationships. In class in the preceding scatterplot matrix, I can see that 4 is a malignant classification. I can also see that there are cells that are scored from 1 to 10 on ``clump_thickness``, and were still classified as malignant. Thus, I come to the conclusion that there aren't any strong relationships between any of the variables of our dataset.  Split the Dataset into $x$ and $y$ Datasets for TrainingThe first step that I need to perform is to split my dataset into $x$ and $y$ datasets for training. I won't train all of the available data, as I need to save some for our validation step. This will help me to determine how well these algorithms can generalize to new data, and not just how well they know the training data. Create the $x$ DatasetMy $x$ data will contain all of the variables, except for the ``class`` column.<jupyter_code># Create the x data
x_and_y_trainer = 'class'
"""
str: The heading for the column I want to drop from the x-dataset and the column I want to use for the y-dataset.
"""
x_dataframe = df.drop([x_and_y_trainer], 1)
"""
pandas.core.frame.DataFrame: The DataFrame with my training data.

It has everything except the class labels.
"""
x_array = np.array(x_dataframe)
"""
numpy.ndarray: The array with my training data.

It has everything except the class labels.
"""
x_array<jupyter_output><empty_output><jupyter_text>Create the $y$ DatasetMy $y$ data (labels) is going to be the ``class`` column, which is the classification of whether a tumor is malignant or benign.<jupyter_code># Create the y data
y_dataframe = df[x_and_y_trainer]
"""
pandas.core.frame.DataFrame: The DataFrame with my training data.

It only has the class labels.
"""
y_array = np.array(y_dataframe)
"""
numpy.ndarray: The array with my training data.

It only has the class labels.
"""
y_array<jupyter_output><empty_output><jupyter_text>Create the Training and Testing Data and LabelsNow I'll create the training data (``x_train``) and training labels (``y_train``), which will be separate from the testing data (``x_test``) and testing labels (``y_test``).<jupyter_code># Create the training and testing datasets
test_size = 0.2
"""
float: The proportion of the dataset to include in the test split.
"""
x_train, x_test, y_train, y_test = model_selection.train_test_split(x_array, y_array, test_size = test_size)
"""
numpy.ndarray: The training and testing data and labels.
"""
print(x_train.shape)
print(y_train.shape)<jupyter_output>(559, 9)
(559,)<jupyter_text>Set Parameters for Training.I'll set both a seed for my random number generator and a scoring criteria.<jupyter_code># Testing options
seed = 8
"""
int: The seed for the random number generator.

I set the seed for reproducability.
"""
scoring = 'accuracy'
"""
str: The scoring criteria for the training.
"""<jupyter_output><empty_output><jupyter_text>I'll make a ``list`` to store the models I'll make.<jupyter_code># Create a list to store the models to train
models = []
"""
list: A list to store my KNN and SVM models.
"""<jupyter_output><empty_output><jupyter_text>Create the K-Nearest Neighbors (KNN) ModelTODO: Finish write-up.<jupyter_code># K-nearest neighbors (KNN)
n_neighbors = 5
"""
int: The number of neighbors to use by default for kneighbors queries.
"""
name_of_knn = 'KNN'
"""
str: The name I'll use to retrieve the k-nearest neighbors (KNN) model.
"""
knn_model = KNeighborsClassifier(n_neighbors = n_neighbors)
"""
sklearn.neighbors._classification.KNeighborsClassifier: The KNN model I'll use for the training.
"""
knn_tuple = (name_of_knn, knn_model)
"""
tuple[str, sklearn.neighbors._classification.KNeighborsClassifier]: A tuple with the name of the KNN model and the KNN model to use for training.

I'll use this tuple to append my models list.
"""
models.append(knn_tuple)<jupyter_output><empty_output><jupyter_text>Create the C-Support Vector Classification (SVC)TODO: Finish write-up<jupyter_code># C-support vector classification (SVC)
name_of_svc = "SVC"
"""
str: The name I'll use to retrieve the C-support vector classification (SVC) model.
"""
svc_model = SVC()
"""
sklearn.svm._classes.SVC: The SVC model I'll use for the training.
"""
svc_tuple = (name_of_svc, svc_model)
"""
tuple[str, sklearn.svm._classes.SVC]: A tuple with the name of the SVC model and the SVC model to use for training.

I'll use this tuple to append my models list.
"""
models.append(svc_tuple)<jupyter_output><empty_output><jupyter_text>Parameters to Evaluate Each Model in Turn<jupyter_code># Store the results and names of the training sessions
results = []
"""
lists: A list to store the accuracy values of the models.
"""
names = []
"""
lists: A list to store the names of the models used for training.
"""<jupyter_output><empty_output><jupyter_text>Do the Training<jupyter_code># Helper function to add training results to the correct list
def _append_training_lists(results_list, results, names_list, names):
    """
    Add the names of the training models and their accuracies per run to their respective list.
    
    Parameters
    ----------
    results_list : list
        A list to store the accuracies of each model.
    results : float
        The accuracy for a particular model.
    names_list : list
        A list to store the names of the models I'll use.
    names : str
        The names of the models I'll use.
    
    Returns
    -------
    None
    """
    results_list.append(results)
    names_list.append(names)
# Helper function to print out the results of training into a readable format
def _print_message(name, cv_results):
    """
    Print the results from model training in a formatted message.
    
    Parameters
    ----------
    name : str
        The name of the model to train.
    cv_results: list of float
        The accuracies for each training run.
    
    Returns
    -------
    None
    """
    message = "%s: %f (%f)" % (name, cv_results.mean(), cv_results.std())
    print(message)
# Perform trainings
n_splits = 10
"""
int: The number of splits to use for the KNN model.
"""

for name, model in models:
    kfold = model_selection.KFold(n_splits = n_splits, shuffle = True, random_state = seed)
    cv_results = model_selection.cross_val_score(estimator = model, X = x_train, y = y_train, cv = kfold, scoring = scoring)
    _append_training_lists(results, cv_results, names, name)
    _print_message(name, cv_results)<jupyter_output>KNN: 0.974968 (0.018198)
SVC: 0.651234 (0.067537)<jupyter_text>Make Predictions on the Validation Dataset<jupyter_code># This helper function will print how well the models can predict results using the validation dataset as a metric.
def _print_validation_message(name, predictions):
    print(name)
    print(accuracy_score(y_test, predictions))
    print(classification_report(y_test, predictions, zero_division = 0))

# Accuracy - ratio of correctly predicted observation to the total observations. 
# Precision - (false positives) ratio of correctly predicted positive observations to the total predicted positive observations
# Recall (Sensitivity) - (false negatives) ratio of correctly predicted positive observations to the all observations in actual class - yes.
# F1 score - F1 Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false
# Make predictions on the validation dataset
for name, model in models:
    model.fit(x_train, y_train)
    predictions = model.predict(x_test)
    _print_validation_message(name, predictions)<jupyter_output>KNN
0.9571428571428572
              precision    recall  f1-score   support

           2       0.98      0.96      0.97        94
           4       0.92      0.96      0.94        46

    accuracy                           0.96       140
   macro avg       0.95      0.96      0.95       140
weighted avg       0.96      0.96      0.96       140

SVC
0.6714285714285714
              precision    recall  f1-score   support

           2       0.67      1.00      0.80        94
           4       0.00      0.00      0.00        46

    accuracy                           0.67       140
   macro avg       0.34      0.50      0.40       140
weighted avg       0.45      0.67      0.54       140
{'fraction_non_alphanumeric': 0.08767806267806268, 'fraction_numerical': 0.01566951566951567, 'mean_word_length': 4.196521095484826, 'pattern_counts': {'":': 0, '<': 89, '<?xml version=': 0, '>': 89, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '339762', 'license': 'MIT', 'n_tokens_mistral': 4420, 'n_tokens_neox': 4166, 'n_words': 1651, 'path': 'Chapter01/chapter_01.ipynb', 'repo_name': 'hestrang1993/Machine-Learning-for-Healthcare-Analytics-Projects', 'size': '14040'}