<jupyter_start><jupyter_text>2: Linear Regression     We use elemental PyTorch to implement linear regression here. However, in most actual applications, abstractions such as nn.Module or nn.Linear are used.  Theoretical Overview $$ H(x) = Wx + b $$ $$ cost(W, b) = \frac{1}{m} \sum^m_{i=1} \left( H(x^{(i)}) - y^{(i)} \right)^2 $$  - $H(x)$: 주어진 $x$ 값에 대해 예측을 어떻게 할 것인가 - $cost(W, b)$: $H(x)$ 가 $y$ 를 얼마나 잘 예측했는가  import libraries<jupyter_code>import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
torch.__version__
torch.manual_seed(1)
import sys
import torch_custom_functions as tcf
sys.version_info<jupyter_output><empty_output><jupyter_text>Low Level Settings data 생성<jupyter_code>x_train = torch.FloatTensor([[1], [2], [3]])
y_train = torch.FloatTensor([[1], [2], [3]])<jupyter_output><empty_output><jupyter_text>weights, bias setting<jupyter_code>w = torch.zeros(1, requires_grad = True)
b = torch.zeros(1, requires_grad = True)<jupyter_output><empty_output><jupyter_text>optimizer setting<jupyter_code>optimizer = optim.SGD([w, b], lr = 0.01)<jupyter_output><empty_output><jupyter_text>gradient descent with 반복문<jupyter_code>epoch = 1000
for i in range(epoch + 1):
    # hypothesis 계산
    h = x_train * w + b
    
    # loss function 계산
    loss = tcf.mse(h, y_train)
    
    # optimizer 설정
    optimizer.zero_grad() # optimzer 초기화 -> pytorch는 누적 방식
    loss.backward() # backpropagation
    optimizer.step() # parameter update
    
    # 100번마다 로그 출력
    if i % 100 == 0:
        print('Epoch {:4d}/{} W: {:.3f}, b: {:.3f} Cost: {:.6f}'.format(
            i, epoch, w.item(), b.item(), loss.item()
        ))<jupyter_output>Epoch    0/1000 W: 0.093, b: 0.040 Cost: 4.666667
Epoch  100/1000 W: 0.873, b: 0.289 Cost: 0.012043
Epoch  200/1000 W: 0.900, b: 0.227 Cost: 0.007442
Epoch  300/1000 W: 0.921, b: 0.179 Cost: 0.004598
Epoch  400/1000 W: 0.938, b: 0.140 Cost: 0.002842
Epoch  500/1000 W: 0.951, b: 0.110 Cost: 0.001756
Epoch  600/1000 W: 0.962, b: 0.087 Cost: 0.001085
Epoch  700/1000 W: 0.970, b: 0.068 Cost: 0.000670
Epoch  800/1000 W: 0.976, b: 0.054 Cost: 0.000414
Epoch  900/1000 W: 0.981, b: 0.042 Cost: 0.000256
Epoch 1000/1000 W: 0.985, b: 0.033 Cost: 0.000158<jupyter_text>High-Level settings data 생성<jupyter_code>x_train = torch.FloatTensor([[1], [2], [3]])
y_train = torch.FloatTensor([[1], [2], [3]])<jupyter_output><empty_output><jupyter_text>PyTorch 모델 생성<jupyter_code>class LinearRegressionModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(1, 1)
    
    def forward(self, x):
        return self.linear(x)
linear_model = LinearRegressionModel()
# list(linear_model.parameters())<jupyter_output><empty_output><jupyter_text>optimizer 설정<jupyter_code>optimizer = optim.SGD(linear_model.parameters(), lr = 0.01)<jupyter_output><empty_output><jupyter_text>gradient descent with 반복문<jupyter_code>epoch = 1000

# linear_model = LinearRegressionModel()
# optimizer = optim.SGD(linear_model.parameters(), lr = 0.01)
for i in range(epoch+1):
    # hypothesis 계산
    prediction = linear_model(x_train)
    
    # loss 계산
    loss = F.mse_loss(prediction, y_train)
    
    # 학습
    optimizer.zero_grad() # optimzer 초기화 -> pytorch는 누적 방식
    loss.backward() # backpropagation
    optimizer.step() # parameter update
    
    # 100번마다 로그 출력
    if i % 100 == 0:
        params = list(linear_model.parameters())
        w = params[0].item()
        b = params[1].item()
        print('Epoch {:4d}/{} W: {:.3f}, b: {:.3f} Cost: {:.6f}'.format(
            i, epoch, w, b, loss.item()
        ))<jupyter_output>Epoch    0/1000 W: 0.578, b: -0.413 Cost: 2.147149
Epoch  100/1000 W: 1.066, b: -0.150 Cost: 0.003239
Epoch  200/1000 W: 1.052, b: -0.118 Cost: 0.002002
Epoch  300/1000 W: 1.041, b: -0.093 Cost: 0.001237
Epoch  400/1000 W: 1.032, b: -0.073 Cost: 0.000764
Epoch  500/1000 W: 1.025, b: -0.057 Cost: 0.000472
Epoch  600/1000 W: 1.020, b: -0.045 Cost: 0.000292
Epoch  700/1000 W: 1.016, b: -0.035 Cost: 0.000180
Epoch  800/1000 W: 1.012, b: -0.028 Cost: 0.000111
Epoch  900/1000 W: 1.010, b: -0.022 Cost: 0.000069
Epoch 1000/1000 W: 1.008, b: -0.017 Cost: 0.000043<jupyter_text>같은 코드인데 위와 결과가 다름 왜일까?(seed설정을 했음에도 불구하고) - jupyter를 나누기 때문에 결과가 달라짐 -> jupyter problem?<jupyter_code># 데이터
x_train = torch.FloatTensor([[1], [2], [3]])
y_train = torch.FloatTensor([[1], [2], [3]])
# 모델 초기화
model = LinearRegressionModel()
# optimizer 설정
optimizer = optim.SGD(model.parameters(), lr=0.01)

nb_epochs = 1000
for epoch in range(nb_epochs + 1):
    
    # H(x) 계산
    prediction = model(x_train)
    
    # cost 계산
    cost = F.mse_loss(prediction, y_train)
    
    # cost로 H(x) 개선
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()
    
    # 100번마다 로그 출력
    if epoch % 100 == 0:
        params = list(model.parameters())
        W = params[0].item()
        b = params[1].item()
        print('Epoch {:4d}/{} W: {:.3f}, b: {:.3f} Cost: {:.6f}'.format(
            epoch, nb_epochs, W, b, cost.item()
        ))<jupyter_output>Epoch    0/1000 W: -0.101, b: 0.508 Cost: 4.630286
Epoch  100/1000 W: 0.713, b: 0.653 Cost: 0.061555
Epoch  200/1000 W: 0.774, b: 0.514 Cost: 0.038037
Epoch  300/1000 W: 0.822, b: 0.404 Cost: 0.023505
Epoch  400/1000 W: 0.860, b: 0.317 Cost: 0.014525
Epoch  500/1000 W: 0.890, b: 0.250 Cost: 0.008975
Epoch  600/1000 W: 0.914, b: 0.196 Cost: 0.005546
Epoch  700/1000 W: 0.932, b: 0.154 Cost: 0.003427
Epoch  800/1000 W: 0.947, b: 0.121 Cost: 0.002118
Epoch  900/1000 W: 0.958, b: 0.095 Cost: 0.001309
Epoch 1000/1000 W: 0.967, b: 0.075 Cost: 0.000809
{'fraction_non_alphanumeric': 0.15655123170296323, 'fraction_numerical': 0.14530524812566942, 'mean_word_length': 3.336687306501548, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '601670', 'license': 'MIT', 'n_tokens_mistral': 2995, 'n_tokens_neox': 2565, 'n_words': 595, 'path': 'Pytorch/pytorch_basic/2_linear_regression.ipynb', 'repo_name': 'IzPerfect/My-dictionary', 'size': '5602'}