<jupyter_start><jupyter_text>Gradient Descent Implementation Challenge!! Use gradient descent to find the optimal parameters of a **multiple** regression model. (We only showed an implementation for a bivariate model during lecture.)Implementing gradient descent in any context is not trivial, particularly the step where we calculate the gradient will change based on the number of parameters that we're trying to optimize for. You will need to research what the gradient of a multiple regression model looks like. This challenge is pretty open-ended but I hope it will be thrilling. Please work together, help each other, share resources and generally expand your understanding of gradient descent as you try and achieve this implementation.  Suggestions:Start off with a model that has just two $X$ variables You can use any datasets that have at least two $x$ variables. Potential candidates might be the blood pressure dataset that we used during lecture on Monday: [HERE](https://college.cengage.com/mathematics/brase/understandable_statistics/7e/students/datasets/mlr/excel/mlr02.xls) or any of the housing datasets. You would just need to select from them the two $x$ variables and one $y$ variable that you want to work with. Use Sklearn to find the optimal parameters of your model first. (like we did during the lecture.) So that you can compare the parameter estimates of your gradient-descent linear regression to the estimates of OLS linear regression. If implemented correctly they should be nearly identical.Becoming a Data Scientist is all about striking out into the unknown, getting stuck and then researching and fighting and learning until you get yourself unstuck. Work together! And fight to take your own learning-rate fueled step towards your own optimal understanding of gradient descent!<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression

plt.style.use('seaborn')

df = pd.read_excel('https://college.cengage.com/mathematics/brase/'
                   'understandable_statistics/7e/students/datasets/'
                   'mlr/excel/mlr02.xls', names=['mmHg', 'age', 'weight'])
y = df[['mmHg']].values
X = df[['age', 'weight']].values

X = (X - X.mean(axis=0)) / X.std(axis=0)

model = LinearRegression().fit(X, y)
print(model.coef_[0], model.intercept_, sep='\n')
def multi_gradient_descent(X, y, theta, iters, alpha):
    n = X.shape[0] # num observations
    m = X.shape[1] # num features
    
    theta = np.array([theta])
    
    # add col of ones
    X = np.c_[np.ones(X.shape[0]), X]
    
    costs = []
    for i in range(iters):
        predict = X @ theta.T
        error = predict - y
        msse = np.sum(error**2) / (2 * n)
        costs.append(msse)
        
        theta = theta - (alpha / m) * np.sum(X * error, axis=0)
        
    return theta, costs

# one extra theta than cols because of intercept term
theta, costs = multi_gradient_descent(
    X, y, theta=[0, 0,0], iters=2000, alpha=0.02
)

print(theta)

plt.plot(costs);<jupyter_output>[[150.09090909   7.48591307   5.52818061]]
{'fraction_non_alphanumeric': 0.08632862644415917, 'fraction_numerical': 0.017329910141206675, 'mean_word_length': 3.8551401869158877, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '120120', 'license': 'MIT', 'n_tokens_mistral': 929, 'n_tokens_neox': 870, 'n_words': 373, 'path': '06-Linear-Regression/03_Gradient_Descent_Assignment.ipynb', 'repo_name': 'shalevy1/data-science-journal', 'size': '3116'}