<jupyter_start><jupyter_text>Import Data<jupyter_code>df = pd.read_csv("data/heart_failure_clinical_records_dataset.csv")
df.head()<jupyter_output><empty_output><jupyter_text>Eksplorasi Dataset<jupyter_code>plt.figure(figsize=(16,12))
plt.title('Correlation Heatmap of Heart Failure')
a = sns.heatmap(df.corr(), square=True, annot=True, fmt='.2f', vmin=-1, vmax=1, linecolor='white', cmap= 'coolwarm')
a.set_xticklabels(a.get_xticklabels(), rotation=90)
a.set_yticklabels(a.get_yticklabels(), rotation=30)           
plt.show()<jupyter_output><empty_output><jupyter_text>bisa kita lihat disini age dan serum creatinine cukup berkorelasi kuat dengan target, maka mungkin keduanya merupakan fitur yang cukup penting dan fitur time berkorelasi lemah dengan target, ini bisa jadi pertimbangan untuk kita drop. Gambar histogram dari `age`<jupyter_code>f, ax = plt.subplots(figsize=(10,6))
x = df["age"]
ax = sns.distplot(x, bins=10)
plt.show()<jupyter_output><empty_output><jupyter_text>Metrik yang kita gunakan untuk uji normalitas ini adalah uji Jarque Bera. Untuk sampel yang lebih besar dari 200 selain menggunakan Jarque Bera, kita juga bisa mempunyai alternatif lain seperti uji Skewness Kurtosis. Tetapi disini saya menggunakan Uji Jarque Bera karena cocok untuk mengukur apakah skewness sesuai dengan distribusi normal.<jupyter_code>stat, p = st.jarque_bera(df.age)
stat, p<jupyter_output><empty_output><jupyter_text>Berdasarkan uji jarque bera, diperoleh p-value = 0.009 < α = 0.05. Dengan menggunakan tingkat kepercayaan 95%, data yang ada mendukung bahwa data tidak berdistribusi normal. Maka nanti data tersebut bisa kita transform. Dengan asumsi bahwa variabel `serum_sodium` terdistribusi Gaussian, maka berapa peluang mengobservasi nilainya lebih dari 140?<jupyter_code>pd.set_option("display.precision", 4)
df.describe()
from scipy.stats import norm

mean = df.serum_sodium.mean() #mean
std = df.serum_sodium.std() #standard deviasi
dist = norm(mean, std) #distribusi gaussian normal atau normal distribution

serums = df.serum_sodium.values
serums.sort()
values = [value for value in serums]
#hitung probabilitynya (zscore)
probabilities = [dist.pdf(value) for value in serums]

plt.plot(values, probabilities);

#filter yg lebih dari 140
serums = df.serum_sodium[df.serum_sodium > 140].unique()
probabilities = [dist.pdf(value) for value in serums]
f"Peluang: {np.sum(probabilities)}"<jupyter_output><empty_output><jupyter_text>cek target<jupyter_code>df["DEATH_EVENT"].value_counts() / len(df)
df["DEATH_EVENT"].unique()
df.groupby("DEATH_EVENT").mean()
#kita lihat distribusi target
f, ax = plt.subplots(figsize=(8, 6))
ax = sns.countplot(x="DEATH_EVENT", data=df)
plt.show()<jupyter_output><empty_output><jupyter_text>Kurang lebih ada setengahnya yang meninggal karena gagal jantung<jupyter_code>#kita lihat hubungan antara sex dan target
f, ax = plt.subplots(figsize=(8, 6))
ax = sns.countplot(x="sex", hue="DEATH_EVENT", data=df)
plt.show()<jupyter_output><empty_output><jupyter_text>kita juga bisa melihat disini pasien laki-laki lebih banyak yang terkena gagal jantung<jupyter_code>#hubungan target dengan age
f, ax = plt.subplots(figsize=(8, 6))
sns.stripplot(x="DEATH_EVENT", y="age", data=df)
plt.show()<jupyter_output><empty_output><jupyter_text>Kita dapat melihat bahwa orang yang terkena gagal jantung dan orang yang tidak terkena gagal jantung memiliki usia yang sebanding.<jupyter_code># check for missing values

df.isnull().sum()<jupyter_output><empty_output><jupyter_text>Kita dapat melihat bahwa tidak ada missing value dalam dataset.  Klasifikasi Bagi dataset menjadi data latih dan data uji dengan rasio 70:30. Buang kolom `time` dari data dan gunakan `DEATH_EVENT` sebagai kelas yang akan diprediksi.   Dataset Splitting<jupyter_code>df.columns
X = df.drop(columns=["DEATH_EVENT", "time"])
y = df.DEATH_EVENT

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, stratify=y, random_state=42)
X_train.shape, X_test.shape, y_train.shape, y_test.shape<jupyter_output><empty_output><jupyter_text>Training<jupyter_code>X_train.head()
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV
from jcopml.tuning import grid_search_params as gsp
preprocessor = ColumnTransformer([
    ('numeric', num_pipe(), ["age", "creatinine_phosphokinase", "ejection_fraction", "platelets", "serum_creatinine", "serum_sodium"]),
    ('categoric', cat_pipe(encoder='onehot'), ["anaemia", "diabetes", "high_blood_pressure", "sex", "smoking"]),
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('algo', LogisticRegression(solver='lbfgs', n_jobs=-1, random_state=42))
])

model = GridSearchCV(pipeline, gsp.logreg_params, cv=3, n_jobs=-1, verbose=1)
model.fit(X_train, y_train)

print(model.best_params_)
print(model.score(X_train, y_train), model.best_score_, model.score(X_test, y_test))<jupyter_output>Fitting 3 folds for each of 14 candidates, totalling 42 fits<jupyter_text>akurasi sudah cukup baik, tetapi masih bisa kita improve:*   tambahkan scaling dan transform datanya*   mencoba algoritma lain*   bisa juga kita binning fitur age nya, mungkin data itu bisa menjadi lebih bermakna<jupyter_code>#coba pakai scaling standard sama transform yeo johnson
preprocessor = ColumnTransformer([
    ('numeric', num_pipe(scaling='standard', transform='yeo-johnson'), ["age", "creatinine_phosphokinase", "ejection_fraction", "platelets", "serum_creatinine", "serum_sodium"]),
    ('categoric', cat_pipe(encoder='onehot'), ["anaemia", "diabetes", "high_blood_pressure", "sex", "smoking"]),
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('algo', LogisticRegression(solver='lbfgs', n_jobs=-1, random_state=42))
])

model = GridSearchCV(pipeline, gsp.logreg_params, cv=3, n_jobs=-1, verbose=1)
model.fit(X_train, y_train)

print(model.best_params_)
print(model.score(X_train, y_train), model.best_score_, model.score(X_test, y_test))
#coba algoritma lain
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import RandomizedSearchCV
from jcopml.tuning import random_search_params as rsp
preprocessor = ColumnTransformer([
    ('numeric', num_pipe(scaling='standard', transform='yeo-johnson', poly=2), ["age", "creatinine_phosphokinase", "ejection_fraction", "platelets", "serum_creatinine", "serum_sodium"]),
    ('categoric', cat_pipe(encoder='onehot'), ["anaemia", "diabetes", "high_blood_pressure", "sex", "smoking"]),
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('algo', RandomForestClassifier(n_jobs=-1, random_state=42))
])

model = RandomizedSearchCV(pipeline, rsp.rf_poly_params, cv=3,  n_iter=50, n_jobs=-1, verbose=1, random_state=42)
model.fit(X_train, y_train)

print(model.best_params_)
print(model.score(X_train, y_train), model.best_score_, model.score(X_test, y_test))<jupyter_output>C:\Users\ASUS\Miniconda3\envs\jcopml\lib\site-packages\jcopml\pipeline\_pipeline.py:65: UserWarning: Transformer has default standardization, so the scaling argument is neglected
  warn("Transformer has default standardization, so the scaling argument is neglected")<jupyter_text>ternyata hasil nya masih lebih bagus pakai algoritma logistic regression, karena jika soal klasifikasi seperti ini lebih cocok logistic regression<jupyter_code>#coba tambahkan poly
preprocessor = ColumnTransformer([
    ('numeric', num_pipe(scaling='standard', transform='yeo-johnson', poly=2), ["age", "creatinine_phosphokinase", "ejection_fraction", "platelets", "serum_creatinine", "serum_sodium"]),
    ('categoric', cat_pipe(encoder='onehot'), ["anaemia", "diabetes", "high_blood_pressure", "sex", "smoking"]),
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('algo', LogisticRegression(solver='lbfgs', n_jobs=-1, random_state=42))
])

model = GridSearchCV(pipeline, gsp.logreg_poly_params, cv=3, n_jobs=-1, verbose=1)
model.fit(X_train, y_train)

print(model.best_params_)
print(model.score(X_train, y_train), model.best_score_, model.score(X_test, y_test))
#coba kita binning dulu age nya
#dimana numeric menjadi categoric, dengan harapan jika jadi categoric akan lebih bermakna data tersebut
#min40, max95
df["age_category"] = pd.cut(df.age, [40, 50, 70, 95], labels=["dewasa", "lansia_awal", "lansia_akhir"])
df.head()
df.age_category.value_counts()
X = df.drop(columns=["DEATH_EVENT", "time"])
y = df.DEATH_EVENT

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, stratify=y, random_state=42)
X_train.shape, X_test.shape, y_train.shape, y_test.shape
#pakai binningnya
preprocessor = ColumnTransformer([
    ('numeric', num_pipe(scaling='standard', transform='yeo-johnson'), ["creatinine_phosphokinase", "ejection_fraction", "platelets", "serum_creatinine", "serum_sodium"]),
    ('categoric', cat_pipe(encoder='onehot'), ["age_category", "anaemia", "diabetes", "high_blood_pressure", "sex", "smoking"]),
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('algo', LogisticRegression(solver='lbfgs', n_jobs=-1, random_state=42))
])

model = GridSearchCV(pipeline, gsp.logreg_params, cv=3, n_jobs=-1, verbose=1)
model.fit(X_train, y_train)

print(model.best_params_)
print(model.score(X_train, y_train), model.best_score_, model.score(X_test, y_test))<jupyter_output>C:\Users\ASUS\Miniconda3\envs\jcopml\lib\site-packages\jcopml\pipeline\_pipeline.py:65: UserWarning: Transformer has default standardization, so the scaling argument is neglected
  warn("Transformer has default standardization, so the scaling argument is neglected")
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.<jupyter_text>ternyata hasilnya tidak cukup baik dari model kedua kita tadi<jupyter_code>preprocessor = ColumnTransformer([
    ('numeric', num_pipe(scaling='standard', transform='yeo-johnson'), ["age", "creatinine_phosphokinase", "ejection_fraction", "platelets", "serum_creatinine", "serum_sodium"]),
    ('categoric', cat_pipe(encoder='onehot'), ["anaemia", "diabetes", "high_blood_pressure", "sex", "smoking"]),
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('algo', LogisticRegression(solver='lbfgs', n_jobs=-1, random_state=42))
])

model = GridSearchCV(pipeline, gsp.logreg_params, cv=3, n_jobs=-1, verbose=1)
model.fit(X_train, y_train)

print(model.best_params_)
print(model.score(X_train, y_train), model.best_score_, model.score(X_test, y_test))
#Evaluation
from jcopml.plot import plot_confusion_matrix
plot_confusion_matrix(X_train, y_train, X_test, y_test, model)
# Feature Importance
df_imp = mean_score_decrease(X_train, y_train, model, plot=True, topk=10)
preprocessor = ColumnTransformer([
    ('numeric', num_pipe(scaling='standard', transform='yeo-johnson'), ["serum_sodium", "ejection_fraction", "serum_creatinine", "creatinine_phosphokinase", "age", "platelets"]),
    ('categoric', cat_pipe(encoder='onehot'), ["high_blood_pressure", "sex"]),
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('algo', LogisticRegression(solver='lbfgs', n_jobs=-1, random_state=42))
])

model = GridSearchCV(pipeline, gsp.logreg_params, cv=3, n_jobs=-1, verbose=1)
model.fit(X_train, y_train)

print(model.best_params_)
print(model.score(X_train, y_train), model.best_score_, model.score(X_test, y_test))<jupyter_output>C:\Users\ASUS\Miniconda3\envs\jcopml\lib\site-packages\jcopml\pipeline\_pipeline.py:65: UserWarning: Transformer has default standardization, so the scaling argument is neglected
  warn("Transformer has default standardization, so the scaling argument is neglected")
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.<jupyter_text>oke ternyata fiture importance tidak cukup membantu disini, maka kita gunakan model dibawah ini.<jupyter_code>preprocessor = ColumnTransformer([
    ('numeric', num_pipe(scaling='standard', transform='yeo-johnson'), ["age", "creatinine_phosphokinase", "ejection_fraction", "platelets", "serum_creatinine", "serum_sodium"]),
    ('categoric', cat_pipe(encoder='onehot'), ["anaemia", "diabetes", "high_blood_pressure", "sex", "smoking"]),
])

pipeline = Pipeline([
    ('prep', preprocessor),
    ('algo', LogisticRegression(solver='lbfgs', n_jobs=-1, random_state=42))
])

model = GridSearchCV(pipeline, gsp.logreg_params, cv=3, n_jobs=-1, verbose=1)
model.fit(X_train, y_train)

print(model.best_params_)
print(model.score(X_train, y_train), model.best_score_, model.score(X_test, y_test))<jupyter_output>C:\Users\ASUS\Miniconda3\envs\jcopml\lib\site-packages\jcopml\pipeline\_pipeline.py:65: UserWarning: Transformer has default standardization, so the scaling argument is neglected
  warn("Transformer has default standardization, so the scaling argument is neglected")
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers.
{'fraction_non_alphanumeric': 0.13317610062893082, 'fraction_numerical': 0.011556603773584905, 'mean_word_length': 7.6714383094751195, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '108050', 'license': 'MIT', 'n_tokens_mistral': 4870, 'n_tokens_neox': 4589, 'n_words': 983, 'path': 'heart_failure.ipynb', 'repo_name': 'edosept/heart_failure_prediction', 'size': '12720'}