<jupyter_start><jupyter_text>4.1 线性回归 4.1.1 标准方程$$\hat{\theta}  = (X^T X)^{-1} X^T y$$在机器学习中，向量通常表示为列向量，是有单一列的二维数组。如果$\Theta$和x为列向量，则预测为$\hat{y}=\Theta^T x$，其中$\Theta^T$为$\Theta$（行向量而不是列向量）的转置，且$\Theta^T x$为$\Theta^T$和$\Theta$的矩阵乘积。<jupyter_code>%matplotlib inline

import numpy as np

import matplotlib.pyplot as plt
X = 2 * np.random.rand(100, 1)
y = 4 + 3 * X + np.random.randn(100, 1)
plt.figure(figsize=(10, 6))
plt.scatter(X, y)
plt.show()
X_b = np.c_[np.ones((100, 1)), X] #在每个实例上add x0=1
theta_best = np.linalg.inv(X_b.T.dot(X_b)).dot(X_b.T).dot(y)
theta_best
X_new = np.array([[0], [2]])
X_new_b = np.c_[np.ones((2, 1)), X_new]
y_predict = X_new_b.dot(theta_best)
y_predict
plt.figure(figsize=(10, 6))
plt.plot(X_new, y_predict, "r--")
plt.plot(X, y, "b.")
plt.axis([0, 2, 0, 15])
plt.show()
from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg.fit(X, y)
lin_reg.intercept_, lin_reg.coef_<jupyter_output><empty_output><jupyter_text>`LinearRegression`类基于`scipy.linalg.lstsq()`函数（最小二乘法）<jupyter_code>theta_best_svd, residuals, rank, s = np.linalg.lstsq(X_b, y, rcond=1e-6)
theta_best_svd<jupyter_output><empty_output><jupyter_text>此函数计算:$$\hat{\theta} = X^+ y$$其中$X^+$是$X$的伪逆，具体说是Moore-Penrose逆，可以使用`np.linalg.pinv()`直接计算这个伪逆<jupyter_code>np.linalg.pinv(X_b).dot(y)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.20434131736526945, 'fraction_numerical': 0.029940119760479042, 'mean_word_length': 7.2024539877300615, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '622044', 'license': 'MIT', 'n_tokens_mistral': 746, 'n_tokens_neox': 729, 'n_words': 85, 'path': 'docs/source/chapter4/4-1_linear_regression.ipynb', 'repo_name': 'ShawnZhang31/ml-handson', 'size': '1336'}