<jupyter_start><jupyter_text>Cardiocascular Disease - CPC-SPP  Content1. Context1. Challenge1. Desenvolvimento de Solução1. Conclusion & Demonstration1. Next steps  1. Context - Monthly results meeting and how much profit was generated.- CFO called for the resolution of a method that can improve predictions.  2. Chagellenge  Trouble- Prediction instability.  Reasons- The current predictions present some divergences.- With instability there are results that generate losses.- There is usually a high cost for making predictions.  Solution- Use machine learning to better predict the diagnoses.- To be able to make predictions through any computer.  3. Solution Development  Data Description<jupyter_code>print('Number of Rows: {}'.format( df1.shape[0]))
print('Number of Cols: {}'.format( df1.shape[1]))
# Central Tendency - mean, meadina 
ct1 = pd.DataFrame(num_attr.apply(np.mean)).T
ct2 = pd.DataFrame(num_attr.apply(np.median)).T

# dispersion - std, min, max, range, skew, kurtosis
d1 = pd.DataFrame(num_attr.apply(np.std)).T 
d2 = pd.DataFrame(num_attr.apply(min)).T 
d3 = pd.DataFrame(num_attr.apply(max)).T 
d4 = pd.DataFrame(num_attr.apply(lambda x: x.max() - x.min())).T 
d5 = pd.DataFrame(num_attr.apply(lambda x: x.skew())).T 
d6 = pd.DataFrame(num_attr.apply(lambda x: x.kurtosis())).T 

# concatenar
m = pd.concat([d2, d3, d4, ct1, ct2, d1, d5, d6]).T.reset_index()
m.columns = ['attributes', 'min', 'max', 'range', 'mean', 'median', 'std', 'skew', 'kurtosis']
m
plt.subplot(2, 2, 1)
sns.histplot(x="height", kde=True, bins=35, data=num_attr);

plt.subplot(2, 2, 2)
sns.histplot(x="weight", kde=True, bins=35, data=num_attr);

plt.subplot(2, 2, 3)
sns.histplot(x="systolic_blood_pressure", kde=True, bins=35, data=num_attr);

plt.subplot(2, 2, 4)
sns.histplot(x="diastolic_blood_pressure", kde=True, bins=35, data=num_attr);
cat_attr.apply(lambda i: i.unique().shape[0])<jupyter_output><empty_output><jupyter_text>MindMap<jupyter_code>Image("../images/mindmap.png")<jupyter_output><empty_output><jupyter_text>Hypothesis 1. The older a person is, the more likely they are to be sick.1. Who smokes has more the chance to be sick.1. Who doesn't make physical activity may be sick.1. The more obese a person is, the more chance of being sick.1. High cholesterol means that the person may have heart problems.  Response Variable<jupyter_code>sns.countplot(x="disease", data=df4);<jupyter_output><empty_output><jupyter_text>Numerical Variable<jupyter_code>ax1 = df4.select_dtypes(exclude=["object"]).drop(columns=["id"], axis=1)
ax1.plot.box();<jupyter_output><empty_output><jupyter_text>Categorical Variables<jupyter_code>aux1 = df4.select_dtypes(include="object")

plt.subplot(3, 3, 1)
sns.countplot(x="gender", data=aux1);

plt.subplot(3, 3, 2)
sns.countplot(x="cholesterol", data=aux1);

plt.subplot(3, 3, 3)
sns.countplot(x="glucose", data=aux1);

plt.subplot(3, 3, 4)
sns.countplot(x="smoking", data=aux1);

plt.subplot(3, 3, 5)
sns.countplot(x="alcohol", data=aux1);

plt.subplot(3, 3, 6)
sns.countplot(x="physical_activity", data=aux1);

plt.subplot(3, 3, 7)
sns.countplot(x="life_stage", data=aux1);

plt.subplot(3, 3, 8)
sns.countplot(x="weight_status", data=aux1);<jupyter_output><empty_output><jupyter_text>Hypotheses Validation  H1 The older a person is, the more likely they are to be sick.**TRUE** The older the pearson is the probability to be sick is higher.<jupyter_code>sns.countplot(x="age_year", data=df4, hue="disease");<jupyter_output><empty_output><jupyter_text>H2 Who smokes has more the chance to be sick.**FALSE** If they smoke it doesn't mean they're dicease.<jupyter_code>sns.countplot(x="smoking", data=df4, hue="disease");<jupyter_output><empty_output><jupyter_text>H3 Who doesn't make physical activity may be sick.**TRUE** There's more sick peaple which doesn't make physical activity.<jupyter_code>sns.countplot(x="physical_activity", data=df4, hue="disease");<jupyter_output><empty_output><jupyter_text>H4 The more obese a person is, the more chance of being sick.**TRUE** There's more sick peaple whom weight status is obese and overweight<jupyter_code>sns.countplot(x="weight_status", data=df4, hue="disease");<jupyter_output><empty_output><jupyter_text>H5 High cholesterol means that the person may have heart problems.**TRUE** There's more sick peaple who has the cholesterol above and well above normal.<jupyter_code>sns.countplot(x="cholesterol", data=df4, hue="disease");<jupyter_output><empty_output><jupyter_text>Multivariate Analysis  Numerical Attributes<jupyter_code>mask = np.zeros_like(corr)
mask[np.triu_indices_from(mask)] = True

with sns.axes_style("white"):
    f, ax = plt.subplots()
    ax = sns.heatmap(corr, annot=True, vmax=1, vmin=-1, mask=mask, center=0, cmap="vlag");<jupyter_output><empty_output><jupyter_text>Categorical Atributes<jupyter_code>cramerv = pd.DataFrame({
    "gender": [c1, c2, c3, c4, c5, c6, c7, c8, c9], 
    "cholesterol": [c10, c11, c12, c13, c14, c15, c16, c17, c18], 
    "glucose": [c19, c20, c21, c22, c23, c24, c25, c26, c27],
    "smoking": [c28, c29, c30, c31, c32, c33, c34, c35, c36],
    "alcohol": [c37, c38, c39, c40, c41, c42, c43, c44, c45],
    "physical_activity": [c46, c47, c48, c49, c50, c51, c52, c53, c54],
    "disease": [c55, c56, c57, c58, c59, c60, c61, c62, c63],
    "life_stage": [c64, c65, c66, c67, c68, c69, c70, c71, c72],
    "weight_status": [c73, c74, c75, c76, c77, c78, c79, c80, c81]
                  })

cramerv = cramerv.set_index(cramerv.columns)

mask = np.zeros_like(cramerv)
mask[np.triu_indices_from(mask)] = True

with sns.axes_style("white"):
    f, ax = plt.subplots()
    ax = sns.heatmap(cramerv, annot=True, vmin=0, vmax=1, mask=mask, cmap="Blues")<jupyter_output><empty_output><jupyter_text>Machine Learning Modelling Compare Model's Performance<jupyter_code>modeling_performance_cv = pd.concat([dummy_cv, lg_cv, knn_cv, rf_cv, xgb_cv])
modeling_performance_cv.sort_values(by="Recall", ascending=True)<jupyter_output><empty_output><jupyter_text>4. Conclusion & Statements  Business Performance<jupyter_code>xgb_resul = ml_scores("XGBoost", y_test, y_pred)
xgb_resul
xgb_cv = ml_cv_results("XGBoost", XGBClassifier(), RobustScaler(), X[best_columns_rf], y)
xgb_cv
aux1 = pd.DataFrame(data={
        "Value": [0.720, 0.722, 0.720], 
        "Best Scenario": [0.720 + 0.003, 0.722 + 0.003, 0.720 + 0.003],
        "Worst Scenario": [0.720 - 0.003, 0.722 - 0.003, 0.720 - 0.003]
}, 
    index=["Accuracy", "Precision", "Recall"])

aux1
accuracy = ["50%", "55%", "60%", "65%", "70%"]
cost = [1000, 1000, 1000, 1000, 1000]
price = [0, 500, 1000, 1500, 2000]

plt.bar(x=accuracy, height=price, label="Price")
plt.bar(x=accuracy, height=cost, color="red", label="Cost", alpha=0.2)
plt.legend(loc="best")
plt.xlabel("Accuracy")
plt.ylabel("R$");<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1567591369440775, 'fraction_numerical': 0.054601497137824744, 'mean_word_length': 6.010288065843621, 'pattern_counts': {'":': 12, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '595216', 'license': 'MIT', 'n_tokens_mistral': 2939, 'n_tokens_neox': 2625, 'n_words': 613, 'path': 'storytelling/storytelling-cicle1-cardiovascular-disease.ipynb', 'repo_name': 'juniorcl/cardiovascular-disease-prediction', 'size': '6813'}