<jupyter_start><jupyter_text>The Data<jupyter_code>data = pd.read_csv("./heart_2020.csv")
data.head(10)
data.info()
print("Number of Datapoints:", len(data))
data.HeartDisease.value_counts().plot.bar()<jupyter_output>Number of Datapoints: 319795<jupyter_text>Data Oversampling – SMOTEThe above chart suggests that the data is severely imbalanced. To overcome this, we will oversample the data using SMOTE.<jupyter_code># Oversample with SMOTE
oversample = SMOTENC(
    categorical_features=[1, 2, 3, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16],
    sampling_strategy='not majority',
    k_neighbors=512
)

data_labels = np.where(data.HeartDisease.to_numpy() == "Yes", 1, 0)
features, labels = oversample.fit_resample(data.to_numpy()[:,1:], data_labels)

print("Number of Datapoints:", len(features))

features_and_labels = np.hstack((features, labels.reshape(-1, 1)))
smote_dataset = pd.DataFrame(features_and_labels, columns=[*data.columns[1:], "HeartDisease"])

smote_dataset<jupyter_output>Number of Datapoints: 584844<jupyter_text>The Model<jupyter_code>clf1 = setup(smote_dataset, target='HeartDisease', session_id=123, experiment_name='hhi1')<jupyter_output><empty_output><jupyter_text>Compare Baselines<jupyter_code>best_model = compare_models()
et_model = create_model('lightgbm')
plot_model(et_model)
plot_model(et_model, plot='confusion_matrix')
plot_model(et_model, plot='boundary')
plot_model(et_model, plot='feature')
plot_model(et_model, plot='class_report')
# finalize the model
finalModel = finalize_model(et_model)
save_model(finalModel, "final_model")
save_config("model_config")<jupyter_output>Transformation Pipeline and Model Successfully Saved
{'fraction_non_alphanumeric': 0.1291490645745323, 'fraction_numerical': 0.030778515389257695, 'mean_word_length': 9.171779141104295, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '430646', 'license': 'Apache-2.0', 'n_tokens_mistral': 648, 'n_tokens_neox': 597, 'n_words': 112, 'path': 'Data/main.ipynb', 'repo_name': 'faustotnc/heart-health-interactive', 'size': '1657'}