<jupyter_start><jupyter_text>15장. 다중 회귀 분석 과제<jupyter_code>!pip install seaborn<jupyter_output>Requirement already satisfied: seaborn in c:\users\pc\anaconda3\envs\data_mining\lib\site-packages (0.11.1)
Requirement already satisfied: pandas>=0.23 in c:\users\pc\anaconda3\envs\data_mining\lib\site-packages (from seaborn) (1.2.3)
Requirement already satisfied: scipy>=1.0 in c:\users\pc\anaconda3\envs\data_mining\lib\site-packages (from seaborn) (1.6.3)
Requirement already satisfied: numpy>=1.15 in c:\users\pc\anaconda3\envs\data_mining\lib\site-packages (from seaborn) (1.20.1)
Requirement already satisfied: matplotlib>=2.2 in c:\users\pc\anaconda3\envs\data_mining\lib\site-packages (from seaborn) (3.3.4)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in c:\users\pc\anaconda3\envs\data_mining\lib\site-packages (from matplotlib>=2.2->seaborn) (2.4.7)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\pc\anaconda3\envs\data_mining\lib\site-packages (from matplotlib>=2.2->seaborn) (1.3.1)
Requirement already satisfied: python-dateutil>=2.1 in c:\user[...]<jupyter_text>1. 데이터셋  1.1 데이터셋 읽기 (Graduate Admission)* 특징 : GRE Scores, TOEFL, CGPA, SOP Rating, LOR Rating, Research Papers, University Ratings* 타겟 : Chance of Admit (대학원 입학 허가 확률)* https://www.kaggle.com/mohansacharya/graduate-admissions<jupyter_code>import os
import pandas as pd

path1 = os.path.join('data', 'Admission_Predict_Ver1.1.csv')
path2 = os.path.join('data', 'Admission_Predict.csv')

data1 = pd.read_csv(path1)
data2 = pd.read_csv(path2)

dataset = pd.concat([data1, data2])

dataset.sample(5)<jupyter_output><empty_output><jupyter_text>1.2 Series No 컬럼 삭제 (Q1)샘플마다 유일하게 존재하는 ID 역할을 하는 컬럼은 학습에 방해가 되므로 삭제하시오.<jupyter_code># your code
dataset = dataset.drop("Serial No.", axis=1)<jupyter_output><empty_output><jupyter_text>2. 데이터 탐색  2.1 요약 통계량<jupyter_code>dataset.describe()<jupyter_output><empty_output><jupyter_text>2.2  누락 데이터 확인<jupyter_code>dataset.isnull().sum()<jupyter_output><empty_output><jupyter_text>2.3 산포도 행렬* 행렬의 대각 방향 : 히스토그램* 행렬의 나머지 셀 : 두 변수 간의 상관성 분석을 위한 산포도<jupyter_code>import seaborn as sns
import matplotlib.pyplot as plt
sns.pairplot(dataset)
plt.show()<jupyter_output><empty_output><jupyter_text>2.4 히트맵<jupyter_code>fig, ax = plt.subplots(figsize=(8, 8))
sns.heatmap(dataset.corr(), linewidths=.5, annot=True, fmt=".2f", cmap='Blues')
plt.title('Graduate Admissions Features Correlations')
plt.show()<jupyter_output><empty_output><jupyter_text>2.5 산포도와 단순 회귀선<jupyter_code>plt.subplots(figsize=(8,6))
sns.regplot(x="GRE Score", y="Chance of Admit ", data=dataset)
plt.subplots(figsize=(8,6))
sns.regplot(x="TOEFL Score", y="Chance of Admit ", data=dataset)
plt.subplots(figsize=(8,6))
sns.regplot(x="CGPA", y="Chance of Admit ", data=dataset)<jupyter_output><empty_output><jupyter_text>3. 데이터 전처리  3.1 입력  및 타겟 데이터 추출<jupyter_code>X = dataset.iloc[:,:-1].values
y = dataset.iloc[:,-1].values # Chance of Admit<jupyter_output><empty_output><jupyter_text>입력 데이터에 상수 항에 대한 입력 1 추가<jupyter_code>X = [[1.0] + list(row[:]) for row in X]<jupyter_output><empty_output><jupyter_text>3.2 데이터셋 분리<jupyter_code>import random
from scratch.machine_learning import train_test_split

random.seed(12)
X_train, X_test, y_train, y_test = train_test_split(X, y, 0.25)
print('train dataset :', len(X_train))
print('test dataset :', len(X_test))<jupyter_output>train dataset : 675
test dataset : 225<jupyter_text>3.3 데이터 표준화 (Q2)훈련 데이터의 평균과 표준 편차로 테스트 데이터를 표준화 하도록 normalization() 함수를 작성해 보시오.<jupyter_code>from scratch.working_with_data import scale
from typing import List
from scratch.linear_algebra import Vector

def normalization(data: List[Vector],
                  means : Vector = None,
                  stdevs : Vector = None) -> List[Vector]:
    # your code
    dim = len(data[0])
    if (means == None or stdevs == None):
        means, stdevs = scale(data)
    
    # Make a copy of each vector
    rescaled = [v[:] for v in data]
    
    for v in rescaled:
        for i in range(dim):
            if stdevs[i] > 0:
                v[i] = (v[i] - means[i]) / stdevs[i]

    return rescaled, means, stdevs
X_train_normed, X_train_means, X_train_stdevs = normalization(X_train)
X_test_normed, _, _  = normalization(X_test, X_train_means, X_train_stdevs)<jupyter_output><empty_output><jupyter_text>4. 선형 회귀 분석 (Linear Regression)  4.1 예측 (Q3)모델 예측 코드를 작성해 보시오.<jupyter_code># your code
from scratch.linear_algebra import dot, Vector

def predict(x: Vector, beta: Vector)-> float:
    return dot(x, beta)<jupyter_output><empty_output><jupyter_text>4.2  손실  함수 (Q4)손실 함수와 그래디언트를 작성해 보시오.<jupyter_code># your code
#잔차 구하는 함수
from typing import List

def error(x:Vector, y:float, beta:Vector) -> float:
    return predict(x,beta) - y

#SE
def squared_error(x: Vector, y: float, beta: Vector) -> float:
    return error(x,y,beta)**2

#그래디언트 계산 함수
def sqerror_gradient(x:Vector, y:float, beta: Vector) -> Vector:
    err = error(x,y,beta)
    return[2*err*x_i for x_i in x]<jupyter_output><empty_output><jupyter_text>4.3 모델 훈련 (Q5)  최소 자승법 선형 회귀의 최소 자승법을 경사 하강법으로 구현하시오.<jupyter_code># your code
import random
import tqdm
from scratch.linear_algebra import vector_mean
from scratch.gradient_descent import gradient_step

def least_squares_fit(xs: List[Vector],
                      ys: List[float],
                      learning_rate: float = 0.001,
                      num_steps: int = 1000,
                      batch_size: int = 1) -> Vector:
    
    #Start with a random guess
    guess = [random.random() for _ in xs[0]]
    
    #num_steps 에폭 반복
    for _ in tqdm.trange(num_steps, desc="least squares fit"):
        #미니 배치 생성
        for start in range(0, len(xs), batch_size):
            batch_xs = xs[start:start+batch_size]
            batch_ys = ys[start:start+batch_size]
            #그래디언트 계산
            gradient = vector_mean([sqerror_gradient(x, y, guess) 
                                    for x, y in zip(batch_xs, batch_ys)])
            #그래디언트 스텝 실행
            guess = gradient_step(guess, gradient, -learning_rate)
    
    return guess
            

learning_rate = 0.001

beta = least_squares_fit(X_train_normed, y_train, learning_rate, 5000, 50)<jupyter_output>least squares fit: 100%|██████████████████████████████████████████████████████████| 5000/5000 [00:31<00:00, 156.92it/s]<jupyter_text>𝜷 확인<jupyter_code>print("beta = ", beta)
plt.plot(beta)
plt.show()<jupyter_output>beta =  [0.7230369263767936, 0.01589118373524194, 0.018101419703239086, 0.008113360737764688, -0.002073350176785202, 0.017606264994132734, 0.0710238708099809, 0.015511699194081541]<jupyter_text>4.4 모델 테스트 (Q6)테스트 데이터를 이용해서 예측을 해보고 SSE를 계산해 보시오.<jupyter_code>def test(xs: List[Vector], ys: List[float], beta : Vector) -> float:

    # your code
    SSE = sum(error(x, y, beta) ** 2for x, y in zip(xs, ys))
    
    pred_y = [predict(xs[i], beta) for i in range(len(xs))]
        
    return pred_y, SSE
pred_y, SSE = test(X_test_normed, y_test, beta)
print("SSE = ", SSE)<jupyter_output>SSE =  0.8671732654574079<jupyter_text>실제 값과 예측 값의 상관 관계<jupyter_code>plt.scatter(y_test, pred_y)
plt.title("Actual vs. Predicted Chance of Admit")
plt.xlabel("Actual")
plt.ylabel("Predict")
plt.show()<jupyter_output><empty_output><jupyter_text>4.5 모델 적합도 (goodness-of-fit)  결정계수<jupyter_code>from scratch.simple_linear_regression import total_sum_of_squares

def multiple_r_squared(xs: List[Vector], ys: Vector, beta: Vector) -> float:
    sum_of_squared_errors = sum(error(x, y, beta) ** 2
                                for x, y in zip(xs, ys))

    return 1.0 - sum_of_squared_errors / total_sum_of_squares(ys)
r_squared = multiple_r_squared(X_test_normed, y_test, beta)
print(r_squared)<jupyter_output>0.8189390601208067
{'fraction_non_alphanumeric': 0.13657467113747743, 'fraction_numerical': 0.04836213567191127, 'mean_word_length': 4.366782006920415, 'pattern_counts': {'":': 0, '<': 72, '<?xml version=': 0, '>': 91, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '660013', 'license': 'MIT', 'n_tokens_mistral': 3665, 'n_tokens_neox': 3711, 'n_words': 647, 'path': '15. Multiple Linear Regression_Homework_Student (1).ipynb', 'repo_name': 'SeolHuiGwan9478/DataMining', 'size': '7754'}