<jupyter_start><jupyter_text>Overview The goal of this project is to find a model that results in a high accuracy rate for determining whether tissue is malignant or benign based off the other data it is given. This technology can be used in hospital to help doctors more accurately determine if somone could have a potentially life-threatening health concern. This is considered a supervised model because we know the two possible outcomes the model can result in- malignant or benign.   Import Data<jupyter_code>import pandas as pd
#Get data set and display it

cancer = pd.read_csv(r'C:\Users\Olivia\Desktop\DATA_601\data\cancer.csv')
cancer.head()
#Benign to Malignant tissue

cancer.loc[:, 'diagnosis'].value_counts()
#Dummy Accuracy Model 
#Ratio of benign to malignant

357/(212 + 357)
#Set y equal to B or M

y = cancer.loc[:, 'diagnosis']
#Display y

y
#Set x to all columns except Diagnosis

x = cancer.loc[:, cancer.columns != 'diagnosis']
#Display x

x
from sklearn.preprocessing import LabelEncoder
#Instantiate

le = LabelEncoder()
#Fit encoder to y

le.fit(y)
#Transform

le.transform(y)
#B is 0, M is 1

le.classes_<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis<jupyter_code>import matplotlib.pyplot as plt
# Display the count, mean, standard deviation, min and max values, as well as the percentiles of the data

cancer.describe()
#Display distribution of B to M

N, bins, patches = plt.hist(y)
for i in range(0,5):
    patches[i].set_facecolor('pink')
for i in range(5,10):    
    patches[i].set_facecolor('lavender')
plt.title('Distribution of Malignant to Benign Breast Tissue')<jupyter_output><empty_output><jupyter_text>Data Cleaning / Prep<jupyter_code># No missing values in data set

x.info()
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.2, random_state = 25, stratify = y )
import numpy as np
np.unique(y_train, return_counts = True)
285/(285 + 170)
np.unique(y_test, return_counts = True)
72/(72 + 42)<jupyter_output><empty_output><jupyter_text>Scale Data<jupyter_code>from sklearn.preprocessing import StandardScaler

std_scaler = StandardScaler()
std_scaler.fit(x_train) 
x_train_s = std_scaler.transform(x_train) 
x_test_s = std_scaler.transform(x_test)
x_train_s<jupyter_output><empty_output><jupyter_text>Feature Engineering<jupyter_code>from sklearn.preprocessing import PolynomialFeatures
poly_feats = PolynomialFeatures(degree= 2, interaction_only= True)
poly_feats.fit(x_train_s)
poly_trained_s = poly_feats.transform(x_train_s)
poly_trained_s.shape
poly_test_s = poly_feats.transform(x_test_s)
poly_test_s.shape<jupyter_output><empty_output><jupyter_text>Data Modeling I will be using a logistic regression model to help determine the results.<jupyter_code>#Import logistic regression

from sklearn.linear_model import LogisticRegression
lr = LogisticRegression(penalty = 'none')
#Fit model

lr.fit(x_train_s, y_train)
y_prediction = lr.predict(x_train_s)
#Setting Score

score = lr.score(x_train_s, y_train)
#Accuracy

score<jupyter_output><empty_output><jupyter_text>Validation<jupyter_code>#Validation

from sklearn.model_selection import cross_validate
lr = LogisticRegression(penalty = 'none')
cross_five = cross_validate(estimator= lr, 
                             X = x_train_s,
                             y = y_train,
                             cv = 5,
                             n_jobs= -1, 
                             return_train_score= True, 
                             return_estimator= True, verbose = 2)
cross_five['test_score']
validation_mean = cross_five['test_score'].mean()
validation_std = cross_five['test_score'].std()
#Print and store results

print('5 fold cross validation results (Accuracy) %.3f =/- %.3f'%(validation_mean, validation_std))<jupyter_output>5 fold cross validation results (Accuracy) 0.960 =/- 0.032
{'fraction_non_alphanumeric': 0.08939121500128436, 'fraction_numerical': 0.013871050603647572, 'mean_word_length': 4.751846381093058, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '13624', 'license': 'MIT', 'n_tokens_mistral': 1350, 'n_tokens_neox': 1262, 'n_words': 358, 'path': 'DATA601_Final.ipynb', 'repo_name': 'Oliviad27/DATA601_Final', 'size': '3893'}