<jupyter_start><jupyter_text>作業在鐵達尼資料集中，今天我們專注觀察變數之間的相關性，以Titanic_train.csv 中，首先將有遺失值的數值刪除，並回答下列問題。* Q1: 透過數值法計算  Age 和  Survived  是否有相關性?* Q2:透過數值法計算  Sex 和 Survived  是否有相關性?* Q3: 透過數值法計算  Age 和  Fare 是否有相關性?   * 提示:    1.產稱一個新的變數 Survived_cate ，資料型態傳換成類別型態      2.把題目中的 Survived 用 Survived_cate 來做分析       3.首先觀察一下這些變數的資料型態後，再來想要以哪一種判斷倆倆的相關性。<jupyter_code># import library
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy import stats
import math
import statistics
import seaborn as sns
from IPython.display import display

import pingouin as pg
import researchpy   
%matplotlib inline<jupyter_output><empty_output><jupyter_text>讀入資料<jupyter_code>df_train = pd.read_csv("Titanic_train.csv")
print(df_train.info())
## 這邊我們做一個調整，把 Survived 變成離散型變數 Survived_cate
df_train['Survived_cate']=df_train['Survived']
df_train['Survived_cate']=df_train['Survived_cate'].astype('object')
print(df_train.info())
display(df_train.head(5))<jupyter_output><empty_output><jupyter_text>Q1: 透過數值法計算  Age 和  Survived  是否有相關性?<jupyter_code>## Age:連續型 Survived_cate 為離散型，所以採用 Eta Squared<jupyter_output><empty_output><jupyter_text>計算相關係數，不能允許有遺失值，所以必須先補值，或者把遺失值刪除<jupyter_code>## 取出資料後，把遺失值刪除
complete_data=df_train[['Age','Survived_cate']].dropna()
display(complete_data)
aov = pg.anova(dv='Age', between='Survived_cate', data=complete_data, detailed=True)
aov
etaSq = aov.SS[0] / (aov.SS[0] + aov.SS[1])
etaSq
def judgment_etaSq(etaSq):
    if etaSq < .01:
        qual = 'Negligible'
    elif etaSq < .06:
        qual = 'Small'
    elif etaSq < .14:
        qual = 'Medium'
    else:
        qual = 'Large'
    return(qual)
judgment_etaSq(etaSq)
g = sns.catplot(x="Survived_cate", y="Age", hue="Survived_cate",
               data=complete_data, kind="violin")<jupyter_output><empty_output><jupyter_text>結論:  年紀和存活沒有相關性(complete_data)，思考是否需要放入模型，或者要深入觀察特性，是否需要做特徵轉換  Q2:透過數值法計算  Sex 和 Survived  是否有相關性?<jupyter_code>## Sex:離散型 Survived_cate 為離散型，所以採用 Cramér's V
contTable = pd.crosstab(df_train['Sex'], df_train['Survived_cate'])
contTable
df = min(contTable.shape[0], contTable.shape[1]) - 1
df
crosstab, res = researchpy.crosstab(df_train['Survived_cate'], df_train['Sex'], test='chi-square')
#print(res)
print("Cramer's value is",res.loc[2,'results'])

#這邊用卡方檢定獨立性，所以採用的 test 參數為卡方 "test =" argument.
# 採用的變數在這個模組中，會自己根據資料集來判斷，Cramer's Phi if it a 2x2 table, or Cramer's V is larger than 2x2.
## 寫一個副程式判斷相關性的強度
def judgment_CramerV(df,V):
    if df == 1:
        if V < 0.10:
            qual = 'negligible'
        elif V < 0.30:
            qual = 'small'
        elif V < 0.50:
            qual = 'medium'
        else:
            qual = 'large'
    elif df == 2:
        if V < 0.07:
            qual = 'negligible'
        elif V < 0.21:
            qual = 'small'
        elif V < 0.35:
            qual = 'medium'
        else:
            qual = 'large'
    elif df == 3:
        if V < 0.06:
            qual = 'negligible'
        elif V < 0.17:
            qual = 'small'
        elif V < 0.29:
            qual = 'medium'
        else:
            qual = 'large'
    elif df == 4:
        if V < 0.05:
            qual = 'negligible'
        elif V < 0.15:
            qual = 'small'
        elif V < 0.25:
            qual = 'medium'
        else:
            qual = 'large'
    else:
        if V < 0.05:
            qual = 'negligible'
        elif V < 0.13:
            qual = 'small'
        elif V < 0.22:
            qual = 'medium'
        else:
            qual = 'large'
    return(qual)

judgment_CramerV(df,res.loc[2,'results'])
g= sns.countplot(x="Sex", hue="Survived_cate", data=df_train)<jupyter_output><empty_output><jupyter_text>數值型態和圖形， 存活和性別存在高度的相關性，要預測存活，一定要把性別加上去。  Q3: 透過數值法計算  Age 和  Fare 是否有相關性?<jupyter_code>## Age 連續 , Fare 連續，用 Pearson 相關係數
## 取出資料後，把遺失值刪除
complete_data=df_train[['Age','Fare']].dropna()
display(complete_data)
# 由於 pearsonr 有兩個回傳結果，我們只需取第一個回傳值為相關係數
corr, _=stats.pearsonr(complete_data['Age'],complete_data['Fare'])
print(corr)
#代表身高和體重有高度線性相關
g = sns.regplot(x="Age", y="Fare", color="g",data=complete_data)
#年齡和身高有關連<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12918080465341736, 'fraction_numerical': 0.018662142510906445, 'mean_word_length': 3.2855659397715473, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '667474', 'license': 'MIT', 'n_tokens_mistral': 2057, 'n_tokens_neox': 2126, 'n_words': 317, 'path': 'Solution/Day_38_Solution.ipynb', 'repo_name': 'YenLinWu/DataScienceMarathon', 'size': '4126'}