<jupyter_start><jupyter_text>Final Project Predict wether a mammogram mass is benign or malignantThe "mammographic masses" public dataset from the UCI repository (source: https://archive.ics.uci.edu/ml/datasets/Mammographic+Mass)This data contains 961 instances of masses detected in mammograms, and contains the following attributes:   1. BI-RADS assessment: 1 to 5 (ordinal)     2. Age: patient's age in years (integer)   3. Shape: mass shape: round=1 oval=2 lobular=3 irregular=4 (nominal)   4. Margin: mass margin: circumscribed=1 microlobulated=2 obscured=3 ill-defined=4 spiculated=5 (nominal)   5. Density: mass density high=1 iso=2 low=3 fat-containing=4 (ordinal)   6. Severity: benign=0 or malignant=1 (binominal)   BI-RADS is an assesment of how confident the severity classification is; The age, shape, margin, and density attributes are the features that to build the model with, and "severity" is the classification to predict based on those attributes.Although "shape" and "margin" are nominal data types, which keras typically doesn't deal with well, they are close enough to ordinal that is not necessary nor ideal discard them. The "shape" for example is ordered increasingly from round to irregular.  Import data<jupyter_code>import pandas as pd
import numpy as np
data = pd.read_csv('..\course_data\mammographic_masses.data.txt', na_values=['?'],
                   names = ['BI-RADS', 'age', 'shape', 'margin', 'density', 'severity'])
# Naming columns, changing ? values for NaN<jupyter_output><empty_output><jupyter_text>Check if there are missing values<jupyter_code>data.isnull().sum()<jupyter_output><empty_output><jupyter_text>Check if there are bias in missing values<jupyter_code>data.loc[(data['age'].isnull()) |
              (data['shape'].isnull()) |
              (data['margin'].isnull()) |
              (data['density'].isnull())]<jupyter_output><empty_output><jupyter_text>Missing values looking pretty random, so just drop'em<jupyter_code>data.dropna(inplace=True)<jupyter_output><empty_output><jupyter_text>Separate features vs labels and change it to numpy arrays<jupyter_code>features = data.iloc[:,1:-1].values
labels = data.iloc[:,-1].values
features, labels<jupyter_output><empty_output><jupyter_text>Scale the data<jupyter_code>from sklearn import preprocessing

scaler = preprocessing.StandardScaler()
features_scaled = scaler.fit_transform(features)<jupyter_output><empty_output><jupyter_text>Separate between training and testing data<jupyter_code>from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(features_scaled, labels, train_size=0.75, random_state=1)<jupyter_output><empty_output><jupyter_text>One hot encode labels<jupyter_code>import tensorflow as tf

y_train = np.asarray(y_train).astype('float32').reshape((-1,1))
y_test = np.asarray(y_test).astype('float32').reshape((-1,1))<jupyter_output><empty_output><jupyter_text>Neural network model<jupyter_code>from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import RMSprop

model = Sequential()
model.add(Dense(4, input_dim=4, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal', activation='sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer=RMSprop(),
              metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Training and checking accuracy<jupyter_code>severity = model.fit(x_train, y_train,
                    epochs=10,
                    verbose=0,
                    validation_data=(x_test, y_test))

score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])<jupyter_output>Test loss: 0.6341714262962341
Test accuracy: 0.754807710647583
{'fraction_non_alphanumeric': 0.11027503892060198, 'fraction_numerical': 0.021536066424494033, 'mean_word_length': 5.810954063604241, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '637536', 'license': 'MIT', 'n_tokens_mistral': 1278, 'n_tokens_neox': 1203, 'n_words': 309, 'path': 'course_projects/MammogramMass_Prediction_ANN.ipynb', 'repo_name': 'leodias-datascience/course-keras_tensorflow_python', 'size': '3854'}