title structured data classification from scratch fchollethttps twitter comfchollet date created 20200609 last modified 20200609 description binary classification of structured data including numerical and categorical features accelerator gpu introduction this example demonstrates how to do structured data classification starting from a raw csv file our data includes both numerical and categorical features we will use keras preprocessing layers to normalize the numerical features and vectorize the categorical ones note that this example should be run with tensorflow 2 5 or higher the dataset our datasethttps archive ics uci edumldatasetsheartdisease is provided by the cleveland clinic foundation for heart disease it s a csv file with 303 rows each row contains information about a patient a sample and each column describes an attribute of the patient a feature we use the features to predict whether a patient has a heart disease binary classification here s the description of each feature column description feature type age age in years numerical sex 1 male 0 female categorical cp chest pain type 0 1 2 3 4 categorical trestbpd resting blood pressure in mm hg on admission numerical chol serum cholesterol in mgdl numerical fbs fasting blood sugar in 120 mgdl 1 true 0 false categorical restecg resting electrocardiogram results 0 1 2 categorical thalach maximum heart rate achieved numerical exang exercise induced angina 1 yes 0 no categorical oldpeak st depression induced by exercise relative to rest numerical slope slope of the peak exercise st segment numerical ca number of major vessels 03 colored by fluoroscopy both numerical categorical thal 3 normal 6 fixed defect 7 reversible defect categorical target diagnosis of heart disease 1 true 0 false target setup tensorflow is the only backend that supports string inputs preparing the data let s download the data and load it into a pandas dataframe the dataset includes 303 samples with 14 columns per sample 13 features plus the target label here s a preview of a few samples the last column target indicates whether the patient has a heart disease 1 or not 0 let s split the data into a training and validation set let s generate tf data dataset objects for each dataframe each dataset yields a tuple input target where input is a dictionary of features and target is the value 0 or 1 let s batch the datasets feature preprocessing with keras layers the following features are categorical features encoded as integers sex cp fbs restecg exang ca we will encode these features using onehot encoding we have two options here use categoryencoding which requires knowing the range of input values and will error on input outside the range use integerlookup which will build a lookup table for inputs and reserve an output index for unkown input values for this example we want a simple solution that will handle out of range inputs at inference so we will use integerlookup we also have a categorical feature encoded as a string thal we will create an index of all possible features and encode output using the stringlookup layer finally the following feature are continuous numerical features age trestbps chol thalach oldpeak slope for each of these features we will use a normalization layer to make sure the mean of each feature is 0 and its standard deviation is 1 below we define 3 utility functions to do the operations encodenumericalfeature to apply featurewise normalization to numerical features encodestringcategoricalfeature to first turn string inputs into integer indices then onehot encode these integer indices encodeintegercategoricalfeature to onehot encode integer categorical features create a normalization layer for our feature prepare a dataset that only yields our feature learn the statistics of the data normalize the input feature create a lookup layer which will turn strings into integer indices prepare a dataset that only yields our feature learn the set of possible string values and assign them a fixed integer index turn the string input into integer indices build a model with this done we can create our endtoend model categorical features encoded as integers categorical feature encoded as string numerical features integer categorical features string categorical features numerical features let s visualize our connectivity graph rankdir lr is to make the graph horizontal train the model we quickly get to 80 validation accuracy inference on new data to get a prediction for a new sample you can simply call model predict there are just two things you need to do 1 wrap scalars into a list so as to have a batch dimension models only process batches of data not single samples 2 call converttotensor on each feature introduction this example demonstrates how to do structured data classification starting from a raw csv file our data includes both numerical and categorical features we will use keras preprocessing layers to normalize the numerical features and vectorize the categorical ones note that this example should be run with tensorflow 2 5 or higher the dataset our dataset https archive ics uci edu ml datasets heart disease is provided by the cleveland clinic foundation for heart disease it s a csv file with 303 rows each row contains information about a patient a sample and each column describes an attribute of the patient a feature we use the features to predict whether a patient has a heart disease binary classification here s the description of each feature column description feature type age age in years numerical sex 1 male 0 female categorical cp chest pain type 0 1 2 3 4 categorical trestbpd resting blood pressure in mm hg on admission numerical chol serum cholesterol in mg dl numerical fbs fasting blood sugar in 120 mg dl 1 true 0 false categorical restecg resting electrocardiogram results 0 1 2 categorical thalach maximum heart rate achieved numerical exang exercise induced angina 1 yes 0 no categorical oldpeak st depression induced by exercise relative to rest numerical slope slope of the peak exercise st segment numerical ca number of major vessels 0 3 colored by fluoroscopy both numerical categorical thal 3 normal 6 fixed defect 7 reversible defect categorical target diagnosis of heart disease 1 true 0 false target setup tensorflow is the only backend that supports string inputs preparing the data let s download the data and load it into a pandas dataframe the dataset includes 303 samples with 14 columns per sample 13 features plus the target label here s a preview of a few samples the last column target indicates whether the patient has a heart disease 1 or not 0 let s split the data into a training and validation set let s generate tf data dataset objects for each dataframe each dataset yields a tuple input target where input is a dictionary of features and target is the value 0 or 1 let s batch the datasets feature preprocessing with keras layers the following features are categorical features encoded as integers sex cp fbs restecg exang ca we will encode these features using one hot encoding we have two options here use categoryencoding which requires knowing the range of input values and will error on input outside the range use integerlookup which will build a lookup table for inputs and reserve an output index for unkown input values for this example we want a simple solution that will handle out of range inputs at inference so we will use integerlookup we also have a categorical feature encoded as a string thal we will create an index of all possible features and encode output using the stringlookup layer finally the following feature are continuous numerical features age trestbps chol thalach oldpeak slope for each of these features we will use a normalization layer to make sure the mean of each feature is 0 and its standard deviation is 1 below we define 3 utility functions to do the operations encode_numerical_feature to apply featurewise normalization to numerical features encode_string_categorical_feature to first turn string inputs into integer indices then one hot encode these integer indices encode_integer_categorical_feature to one hot encode integer categorical features create a normalization layer for our feature prepare a dataset that only yields our feature learn the statistics of the data normalize the input feature create a lookup layer which will turn strings into integer indices prepare a dataset that only yields our feature learn the set of possible string values and assign them a fixed integer index turn the string input into integer indices build a model with this done we can create our end to end model categorical features encoded as integers categorical feature encoded as string numerical features integer categorical features string categorical features numerical features let s visualize our connectivity graph rankdir lr is to make the graph horizontal train the model we quickly get to 80 validation accuracy inference on new data to get a prediction for a new sample you can simply call model predict there are just two things you need to do 1 wrap scalars into a list so as to have a batch dimension models only process batches of data not single samples 2 call convert_to_tensor on each feature
import os


os.environ["KERAS_BACKEND"] = "tensorflow"

import tensorflow as tf
import pandas as pd
import keras
from keras import layers



file_url = (
    "http://storage.googleapis.com/download.tensorflow.org/data/heart.csv"
)
dataframe = pd.read_csv(file_url)



dataframe.shape



dataframe.head()



val_dataframe = dataframe.sample(frac=0.2, random_state=1337)
train_dataframe = dataframe.drop(val_dataframe.index)

print(
    f"Using {len(train_dataframe)} samples for training "
    f"and {len(val_dataframe)} for validation"
)




def dataframe_to_dataset(dataframe):
    dataframe = dataframe.copy()
    labels = dataframe.pop("target")
    ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
    ds = ds.shuffle(buffer_size=len(dataframe))
    return ds


train_ds = dataframe_to_dataset(train_dataframe)
val_ds = dataframe_to_dataset(val_dataframe)



for x, y in train_ds.take(1):
    print("Input:", x)
    print("Target:", y)



train_ds = train_ds.batch(32)
val_ds = val_ds.batch(32)




def encode_numerical_feature(feature, name, dataset):
    
    normalizer = layers.Normalization()

    
    feature_ds = dataset.map(lambda x, y: x[name])
    feature_ds = feature_ds.map(lambda x: tf.expand_dims(x, -1))

    
    normalizer.adapt(feature_ds)

    
    encoded_feature = normalizer(feature)
    return encoded_feature


def encode_categorical_feature(feature, name, dataset, is_string):
    lookup_class = layers.StringLookup if is_string else layers.IntegerLookup
    
    lookup = lookup_class(output_mode="binary")

    
    feature_ds = dataset.map(lambda x, y: x[name])
    feature_ds = feature_ds.map(lambda x: tf.expand_dims(x, -1))

    
    lookup.adapt(feature_ds)

    
    encoded_feature = lookup(feature)
    return encoded_feature





sex = keras.Input(shape=(1,), name="sex", dtype="int64")
cp = keras.Input(shape=(1,), name="cp", dtype="int64")
fbs = keras.Input(shape=(1,), name="fbs", dtype="int64")
restecg = keras.Input(shape=(1,), name="restecg", dtype="int64")
exang = keras.Input(shape=(1,), name="exang", dtype="int64")
ca = keras.Input(shape=(1,), name="ca", dtype="int64")


thal = keras.Input(shape=(1,), name="thal", dtype="string")


age = keras.Input(shape=(1,), name="age")
trestbps = keras.Input(shape=(1,), name="trestbps")
chol = keras.Input(shape=(1,), name="chol")
thalach = keras.Input(shape=(1,), name="thalach")
oldpeak = keras.Input(shape=(1,), name="oldpeak")
slope = keras.Input(shape=(1,), name="slope")

all_inputs = [
    sex,
    cp,
    fbs,
    restecg,
    exang,
    ca,
    thal,
    age,
    trestbps,
    chol,
    thalach,
    oldpeak,
    slope,
]


sex_encoded = encode_categorical_feature(sex, "sex", train_ds, False)
cp_encoded = encode_categorical_feature(cp, "cp", train_ds, False)
fbs_encoded = encode_categorical_feature(fbs, "fbs", train_ds, False)
restecg_encoded = encode_categorical_feature(
    restecg, "restecg", train_ds, False
)
exang_encoded = encode_categorical_feature(exang, "exang", train_ds, False)
ca_encoded = encode_categorical_feature(ca, "ca", train_ds, False)


thal_encoded = encode_categorical_feature(thal, "thal", train_ds, True)


age_encoded = encode_numerical_feature(age, "age", train_ds)
trestbps_encoded = encode_numerical_feature(trestbps, "trestbps", train_ds)
chol_encoded = encode_numerical_feature(chol, "chol", train_ds)
thalach_encoded = encode_numerical_feature(thalach, "thalach", train_ds)
oldpeak_encoded = encode_numerical_feature(oldpeak, "oldpeak", train_ds)
slope_encoded = encode_numerical_feature(slope, "slope", train_ds)

all_features = layers.concatenate(
    [
        sex_encoded,
        cp_encoded,
        fbs_encoded,
        restecg_encoded,
        exang_encoded,
        slope_encoded,
        ca_encoded,
        thal_encoded,
        age_encoded,
        trestbps_encoded,
        chol_encoded,
        thalach_encoded,
        oldpeak_encoded,
    ]
)
x = layers.Dense(32, activation="relu")(all_features)
x = layers.Dropout(0.5)(x)
output = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(all_inputs, output)
model.compile("adam", "binary_crossentropy", metrics=["accuracy"])




keras.utils.plot_model(model, show_shapes=True, rankdir="LR")



model.fit(train_ds, epochs=50, validation_data=val_ds)





sample = {
    "age": 60,
    "sex": 1,
    "cp": 1,
    "trestbps": 145,
    "chol": 233,
    "fbs": 1,
    "restecg": 2,
    "thalach": 150,
    "exang": 0,
    "oldpeak": 2.3,
    "slope": 3,
    "ca": 0,
    "thal": "fixed",
}

input_dict = {
    name: tf.convert_to_tensor([value]) for name, value in sample.items()
}
predictions = model.predict(input_dict)

print(
    f"This particular patient had a {100 * predictions[0][0]:.1f} "
    "percent probability of having a heart disease, "
    "as evaluated by our model."
)