<jupyter_start><jupyter_text>회귀분석  1. 다중선형회귀![image.png](attachment:image.png)<jupyter_code>import pandas as pd
import numpy as np
import statsmodels.api as sm
np.random.seed(2)
u = np.random.rand(10) * 11        # 0 ~ 11
v = (np.random.rand(10) * 11) + 11 # 11 ~ 22
w = (np.random.rand(10) * 29) + 1  # 1 ~ 30
y = 3 + 0.1*u + 2*v - 3*w + np.random.rand(10) * 0.1

data = {'y':y, 'u':u, 'v':v, 'w':w}
df = pd.DataFrame(data)
df
# 상수항 추가
X = sm.add_constant(df[['u','v','w']])

# OLS 검정
multi_model = sm.OLS(df[['y']], X)
fitted_multi_model = multi_model.fit()
fitted_multi_model.summary()<jupyter_output>C:\Anaconda3\lib\site-packages\scipy\stats\stats.py:1535: UserWarning: kurtosistest only valid for n>=20 ... continuing anyway, n=10
  "anyway, n=%i" % int(n))<jupyter_text>2. 식이요법을 적용한 닭 데이터![image.png](attachment:image.png)<jupyter_code>ChickWeight = pd.read_csv('ChickWeight.csv')
ChickWeight.head()
Chick = ChickWeight[ChickWeight['Chick']==1]
import statsmodels.formula.api as smf

result = smf.ols(formula='weight ~ Time', data=Chick).fit()
result.summary()<jupyter_output>C:\Anaconda3\lib\site-packages\scipy\stats\stats.py:1535: UserWarning: kurtosistest only valid for n>=20 ... continuing anyway, n=12
  "anyway, n=%i" % int(n))<jupyter_text>3. cars 데이터* dist가 speed의 변화에 따라 어떻게 변화하는지 회귀분석![image.png](attachment:image.png)<jupyter_code>cars = pd.read_csv('cars.csv')
cars.head()
cars['speed2'] = cars['speed'] **2
cars.head()
import statsmodels.formula.api as smf

result = smf.ols(formula='dist ~ speed+speed2', data=cars).fit()
result.summary()<jupyter_output><empty_output><jupyter_text>4. 적절한 회귀 모형 찾기(산점도 & 잔차도 활용)![image.png](attachment:image.png)<jupyter_code>x = [1,2,3,4,5,6,7,8,9]
y = [5,3,2,3,4,6,10,12,18]
data = {'x' : x, 'y' : y}
df1 = pd.DataFrame(data)
df1<jupyter_output><empty_output><jupyter_text>pandas 산점도<jupyter_code>df1.plot.scatter(x='x', y='y')<jupyter_output><empty_output><jupyter_text>seaborn 산점도<jupyter_code>import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline
sns.scatterplot(x='x', y='y', data=df1)
df2 = df1.copy()
df2['x2'] = df2['x']**2
df2<jupyter_output><empty_output><jupyter_text>* df1<jupyter_code>import statsmodels.formula.api as smf

result1 = smf.ols(formula='y ~ x', data=df1).fit()
result1.summary()
fitted = result1.predict(df1['x'])
residual = df1['y'] - fitted

sns.regplot(fitted, residual, lowess=True, line_kws={'color': 'red'})
plt.plot([fitted.min(), fitted.max()], [0, 0], '--', color='grey')<jupyter_output><empty_output><jupyter_text>* df2<jupyter_code>result2 = smf.ols(formula='y ~ x+x2', data=df2).fit()
result2.summary()
fitted = result2.predict(df2[['x', 'x2']])
residual = df2['y'] - fitted

sns.regplot(fitted, residual, lowess=True, line_kws={'color': 'red'})
plt.plot([fitted.min(), fitted.max()], [0, 0], '--', color='grey')<jupyter_output><empty_output><jupyter_text>5. 최적회귀방적식 선택 : 설명변수 선택![image.png](attachment:image.png)<jupyter_code>data = {'x1':[7,1,11,11,7,11,3,1,2,21,1,11,10],
        'x2':[26,29,56,31,52,55,71,31,54,47,40,66,68],
        'x3':[6,15,8,8,6,9,17,22,18,4,23,9,8],
        'x4':[60,52,20,47,33,22,6,44,22,26,34,12,12],
        'y':[78.5,74.3,104.3,87.6,95.9,109.2,102.7,72.5,93.1,115.9,83.8,113.3,109.4]}
df = pd.DataFrame(data)
df.head()
feature_columns = df.columns.difference(["y"]) # Target column 빼기
X = df[feature_columns].copy()
y = df.y

X = sm.add_constant(X)
X.head()<jupyter_output><empty_output><jupyter_text>A. 전진선택법<jupyter_code>def processSubset(feature_set):
    # Fit model on feature_set and calculate RSS
    model = sm.OLS(y, X[list(feature_set)])
    result = model.fit()
    aic = result.aic
    return {"model":result, "AIC":aic}

def forward(predictors):
    # predictor - 현재 선택되어있는 변수
    # 데이터 변수들이 미리정의된 predictors에 있는지 없는지 확인 및 분류
    
    # Pull out predictors we still need to process
    remaining_predictors = [p for p in X.columns.difference(['const']) if p not in predictors]
    
    results = []
    for p in remaining_predictors:
        results.append(processSubset(predictors+[p]+['const']))
    
    # Wrap everything up in a nice dataframe
    models = pd.DataFrame(results)
    
    # Choose the model with the highest RSS
    best_model = models.loc[models['AIC'].argmin()]
        
    # Return the best model, along with some other useful information about the model
    return best_model


models_fwd = pd.DataFrame(columns=["AIC", "model"])

predictors = []
for i in range(1,len(X.columns.difference(['const']))+1):    
    result = forward(predictors)
    if i>1:
        if result['AIC'] > aic_before:
            break
    models_fwd.loc[i] = result
    predictors = models_fwd.loc[i]["model"].model.exog_names
    aic_before = models_fwd.loc[i]["AIC"]
    predictors = [k for k in predictors if k != 'const']
models_fwd
models_fwd.iloc[-1]["model"].summary()<jupyter_output>C:\Anaconda3\lib\site-packages\scipy\stats\stats.py:1535: UserWarning: kurtosistest only valid for n>=20 ... continuing anyway, n=13
  "anyway, n=%i" % int(n))<jupyter_text>B. 후진제거법<jupyter_code>import itertools

def backward(predictors):
    results = []
    
    for combo in itertools.combinations(predictors, len(predictors)-1):
        results.append(processSubset(list(combo)+['const']))
    
    # Wrap everything up in a nice dataframe
    models = pd.DataFrame(results)
    
    # Choose the model with the highest RSS
    best_model = models.loc[models['AIC'].argmin()]
    
    # Return the best model, along with some other useful information about the model
    return best_model

models_back = pd.DataFrame(columns=["AIC","model"], index = range(1,len(X.columns)))

predictors = X.columns.difference(['const'])
aic_before = processSubset(predictors)['AIC']
                       
while (len(predictors) > 1):
    result = backward(predictors=predictors)
    if result['AIC'] > aic_before :
        break
    models_back.loc[len(predictors) -1] = result
    predictors = models_back.loc[len(predictors) - 1]['model'].model.exog_names
    Bmodel_before = result["AIC"]
    predictors = [k for k in predictors if k != 'const']
models_back
models_back.loc[3]["model"].summary()<jupyter_output>C:\Anaconda3\lib\site-packages\scipy\stats\stats.py:1535: UserWarning: kurtosistest only valid for n>=20 ... continuing anyway, n=13
  "anyway, n=%i" % int(n))<jupyter_text>C. 단계적 선택법<jupyter_code>def Stepwise():
    Stepmodels = pd.DataFrame(columns = ["AIC","model"])
    predictors = []
    aic_before = processSubset(predictors + ['const'])['AIC']
    
    # 변수 1~10개 0-9 -> 1-10
    for i in range(1,len(X.columns.difference(['const']))+1) :
        Forward_result = forward(predictors = predictors) # constant added
        print('forward')
        Stepmodels.loc[i] = Forward_result
        predictors = Stepmodels.loc[i]['model'].model.exog_names
        predictors = [k for k in predictors if k != 'const']
        Backward_result = backward(predictors = predictors)
        if Backward_result["AIC"] < Forward_result["AIC"]:
            Stepmodels.loc[i] = Backward_result
            predictors = Stepmodels.loc[i]["model"].model.exog_names
            aic_before = Stepmodels.loc[i]["AIC"]
            predictors = [k for k in predictors if k != 'const']
            print('backward')
        if Stepmodels.loc[i]["AIC"] > aic_before:
            break
        else :
            Smodel_before = Stepmodels.loc[i]["AIC"]

    return (Stepmodels["model"][len(Stepmodels["model"])])

Stepwise_best_model = Stepwise()
Stepwise_best_model.summary()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16889831631976668, 'fraction_numerical': 0.03831366830173671, 'mean_word_length': 4.373219373219373, 'pattern_counts': {'":': 2, '<': 46, '<?xml version=': 0, '>': 55, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '53883', 'license': 'MIT', 'n_tokens_mistral': 3165, 'n_tokens_neox': 3120, 'n_words': 530, 'path': '[RtoPy]회귀분석.ipynb', 'repo_name': 'kamzzang/ADPStudy', 'size': '7543'}