<jupyter_start><jupyter_text>Chapter 4: Training Models<jupyter_code>import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Two ways to train a linear regression model - 1. "Closed Form" equation    - Directly computes the model parameters that best fir the model to the training set    - aka, parameters that minimize the cost function over the training set - 2. "Gradient Descent"    - Iterative optimazation approach    - Gradually tweaking the parameters to minimize the cost function over the training set    - Eventually converges to the same set of parameters as the first method  Linear Regression  Normal Equation - Creating linear looking data<jupyter_code>np.random.rand()
# Random X's from 0 to 2
X = 2 * np.random.rand(100, 1)
# Slope is 3
# X intercept is 4
y = 4 + 3 * X + np.random.randn(100, 1)
plt.scatter(X, y)<jupyter_output><empty_output><jupyter_text>- Goal is to compute the slope, or theta ![alt text](./images/chapter_4/Normal_equation1.png "Title")<jupyter_code># combines an array of ones with the X's
X_b = np.c_[np.ones((100, 1)), X]
X_b[:10]<jupyter_output><empty_output><jupyter_text>- Making an vector that's only one so that the dot product doesn't change the X<jupyter_code>X_b[:10].T
# Getting the inverse of the dot product of X and its transpose with the dot product of the transpose of X and y
theta_best = np.linalg.inv(X_b.T.dot(X_b)).dot(X_b.T).dot(y)
theta_best<jupyter_output><empty_output><jupyter_text>Making predictions using theta<jupyter_code># new x's that encompass the whole x coordinate
X_new = np.array([[0], [2]])
X_new_b = np.c_[np.ones((2, 1)), X_new]
X_new_b
y_predict = X_new_b.dot(theta_best)
y_predict<jupyter_output><empty_output><jupyter_text>Plotting the Predictions<jupyter_code>plt.plot(X_new, y_predict, "r-")
plt.plot(X, y, "b.")
plt.axis([0, 2, 0, 15])
plt.show()<jupyter_output><empty_output><jupyter_text>Comparing it to Linear Regression<jupyter_code>from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg.fit(X, y)
lin_reg.coef_
lin_reg.intercept_<jupyter_output><empty_output><jupyter_text>- Same as normaliztion<jupyter_code>lin_reg.predict(X_new)<jupyter_output><empty_output><jupyter_text>- `LinearRegression` is based on `scipy.linalg.lstsq()` (least squares)<jupyter_code>theta_best_svd, residuals, rank, s = np.linalg.lstsq(X_b, y, rcond=1e-6)
theta_best_svd
residuals
rank
s<jupyter_output><empty_output><jupyter_text>- Theta is calculated by the dot product of X+(pseudoinverse of X) and y - `np.linalg.pinv()` computes the pseudoinverse<jupyter_code>np.linalg.pinv(X_b).dot(y)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10943963896201579, 'fraction_numerical': 0.014667168108311396, 'mean_word_length': 6.247956403269755, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '615207', 'license': 'Apache-2.0', 'n_tokens_mistral': 1031, 'n_tokens_neox': 957, 'n_words': 260, 'path': 'notebooks/chapter_4_training_models.ipynb', 'repo_name': 'ryoiwata/handson-ml', 'size': '2659'}