<jupyter_start><jupyter_text>Domain – Health Carefocus – Cancer detectionBusiness challenge/requirement<NAME> (JCH) is a leading cancer hospital in USA. It specializes in preventing breast cancer.Over the period of the last few years, JCH has collected breast cancer data from patients who came for screening/treatment.However, this data has almost 30 attributes and is difficult to run and interpret the result. You as ML expert has to reduce the no. of attributes (Dimensionality Reduction) so that results are meaningful and accurate.Key issuesReduce the no. of attributes/features in data to make the results and analysis comprehensible by doctorsConsiderationsNONEData volume- Approx 569 records – file breast-cancer-data.csvFields in Data• Details in Module8-Dimensionality-Reduction-PCA.ipynb notebookAdditional information- NABusiness benefitsImproved success rate of cancer detection and hence direct impact on revenue and profit of hospital. More than that it contributes to JCH's mission "Better Life"<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
breastcancerdata = pd.read_csv('breast-cancer-data.csv')
breastcancerdata.shape
breastcancerdata.info()
breastcancerdata.head()
plt.figure(figsize=(16,10))
sns.heatmap(breastcancerdata.corr(),annot=True)
breastcancerdata.diagnosis.unique()
from sklearn.preprocessing import LabelEncoder
encoder = LabelEncoder()
breastcancerdata['diagnosis'] = encoder.fit_transform(breastcancerdata['diagnosis'].astype('str'))
breastcancerdata.diagnosis.unique()
X = breastcancerdata.drop('diagnosis',axis=1)
y = breastcancerdata.diagnosis
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y,random_state=5, test_size=0.2)
logreg= LogisticRegression(random_state=5)
logreg.fit(X_train,y_train)
lrprediction = logreg.predict(X_test)
accuracy_score(y_test, lrprediction)
from sklearn.decomposition import PCA
pca = PCA(n_components=5)
pca.fit(X)
pca.explained_variance_ratio_*100
plt.plot(pca.explained_variance_ratio_*100)
dftransformedX = pd.DataFrame(pca.transform(X))
dftransformedX.sample(5)
X_train1, X_test1, y_train1, y_test1 = train_test_split(dftransformedX, y, test_size=0.2)
logreg.fit(X_train1,y_train1)
predictionafterpca = logreg.predict(X_test1)
accuracy_score(y_test1, predictionafterpca)
from sklearn.metrics import confusion_matrix
confusion_matrix(y_test1, predictionafterpca)
sns.countplot(predictionafterpca)
sns.countplot(y_test1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07663125948406677, 'fraction_numerical': 0.013277693474962063, 'mean_word_length': 7.878787878787879, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '537263', 'license': 'MIT', 'n_tokens_mistral': 854, 'n_tokens_neox': 806, 'n_words': 226, 'path': 'Cls5-Dimentionality Reduction/DimensionalityReduction-CaseStudy3-solution.ipynb', 'repo_name': 'tuhinssam/MLResources', 'size': '2650'}