<jupyter_start><jupyter_text>Intro to Python: Exercise 4  Linear regression with one variable In the first part of the exercise, we're tasked with implementing linear regression with one variable to predict profits for a food truck. Suppose you are the CEO of a restaurant franchise and are considering different cities for opening a new outlet. The chain already has trucks in various cities and you have data for profits and populations from the cities. Let's start by importing some libraries and examining the data.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import os<jupyter_output><empty_output><jupyter_text>Read the data from the CSV file using Panda library<jupyter_code>!git clone https://github.com/vita-epfl/DLAV-2022.git
path = os.getcwd() + '/DLAV-2022/homeworks/hw1/data/ex1data1.txt'
data = pd.read_csv(path, header=None, names=['Population', 'Profit'])
data.head()
data.describe()<jupyter_output><empty_output><jupyter_text>Let's plot it to get a better idea of what the data looks like.<jupyter_code>data.plot(kind='scatter', x='Population', y='Profit', figsize=(12,8))<jupyter_output><empty_output><jupyter_text>Now let's implement linear regression using gradient descent to minimize the cost function.  The equations implemented in the following code samples are detailed in "ex1.pdf" in the "exercises" folder. First we'll create a function to compute the cost of a given solution (characterized by the parameters theta). The cost function is the Mean Sqaured error in matrix form: $$ MSE(\theta) = \frac{1}{N}\sum_n^N [ y_n-x_n^T*\theta]^2 $$where $\theta$ and $x_n$ are vectors__Hint__: Use the matrix form of the cost function and make use of numpy operations.<jupyter_code>def computeCost(x, y, theta):
    # Compute the cost function
    return None<jupyter_output><empty_output><jupyter_text>Let's add a column of ones to the training set so we can use a vectorized solution to computing the cost and gradients.<jupyter_code>data.insert(0, 'Ones', 1)<jupyter_output><empty_output><jupyter_text>Now let's do some variable initialization.<jupyter_code># set X (training data) and y (target variable)
cols = data.shape[1]
X = data.iloc[:,0:cols-1]
y = data.iloc[:,cols-1:cols]<jupyter_output><empty_output><jupyter_text>Let's take a look to make sure X (training set) and y (target variable) look correct.<jupyter_code>X.head()
y.head()<jupyter_output><empty_output><jupyter_text>Convert X and Y to numpy array for better manipulation. Initiliaze Theta.<jupyter_code>X = np.array(X.values)
y = np.array(y.values).flatten()
theta = np.array([0,0])
theta<jupyter_output><empty_output><jupyter_text>Let's take a quick look at the shape of our matrices.<jupyter_code>X.shape, theta.shape, y.shape<jupyter_output><empty_output><jupyter_text>Now let's compute the cost for our initial solution (0 values for theta).<jupyter_code>computeCost(X, y, theta)<jupyter_output><empty_output><jupyter_text>So far so good.  Now we need to define a function to perform gradient descent on the parameters theta using the update rules. Write first a function that computes the gradient of a matrix and then use it in the gradientDescent function.The gradient descent formula is:$$\theta^{t+1} = \theta^{t} - \alpha*\nabla MSE(\theta^{t})$$where $\nabla MSE(\theta^{t})$ is the gradient of the cost function at $\theta^{t}$__Hint__: Use the matrix form of the gradient and make use of numpy operations.<jupyter_code>def compute_gradient(y, tx, w):
    """Compute the gradient."""
    # ***************************************************
    # INSERT YOUR CODE HERE
    # TODO: compute gradient and loss
    # ***************************************************
    raise NotImplementedError
def gradientDescent(X, y, theta, alpha,max_iters):
    """Gradient descent algorithm."""
    # Define parameters to store w and loss
    ws = [theta]
    cost = np.zeros(max_iters)
    for n_iter in range(max_iters):
        # ***************************************************
        # INSERT YOUR CODE HERE
        # TODO: compute gradient and loss
        # ***************************************************
        raise NotImplementedError
        # ***************************************************
        # INSERT YOUR CODE HERE
        # TODO: update theta by gradient
        # ***************************************************
        raise NotImplementedError
        # store w and loss
        ws.append(theta)
        cost[n_iter] = loss
        print("Gradient Descent({bi}/{ti}): loss={l}, w0={w0}, w1={w1}".format(
              bi=n_iter, ti=max_iters - 1, l=loss, w0=theta[0], w1=theta[1]))

    return theta, cost<jupyter_output><empty_output><jupyter_text>Initialize some additional variables - the learning rate alpha, and the number of iterations to perform.<jupyter_code>alpha = 0.01
iters = 1000<jupyter_output><empty_output><jupyter_text>Now let's run the gradient descent algorithm to fit our parameters theta to the training set.<jupyter_code>g, cost = gradientDescent(X, y, theta, alpha, iters)
g<jupyter_output><empty_output><jupyter_text>Finally we can compute the cost (error) of the trained model using our fitted parameters.<jupyter_code>computeCost(X, y, g)<jupyter_output><empty_output><jupyter_text>Now let's plot the linear model along with the data to visually see how well it fits.<jupyter_code>x = np.linspace(data.Population.min(), data.Population.max(), 100)
f = g[0] + (g[1] * x)

fig, ax = plt.subplots(figsize=(12,8))
ax.plot(x, f, 'r', label='Prediction')
ax.scatter(data.Population, data.Profit, label='Traning Data')
ax.legend(loc=2)
ax.set_xlabel('Population')
ax.set_ylabel('Profit')
ax.set_title('Predicted Profit vs. Population Size')<jupyter_output><empty_output><jupyter_text>Looks pretty good!  Since the gradient decent function also outputs a vector with the cost at each training iteration, we can plot that as well.  Notice that the cost always decreases - this is an example of a convex optimization problem.<jupyter_code>fig, ax = plt.subplots(figsize=(12,8))
ax.plot(np.arange(iters), cost, 'r')
ax.set_xlabel('Iterations')
ax.set_ylabel('Cost')
ax.set_title('Error vs. Training Epoch')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14859887910328262, 'fraction_numerical': 0.008967173738991193, 'mean_word_length': 5.393039918116684, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 65, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '178550', 'license': 'MIT', 'n_tokens_mistral': 1968, 'n_tokens_neox': 1871, 'n_words': 626, 'path': 'homeworks/hw1/Exercise 4_Linear_Regression.ipynb', 'repo_name': 'Jaakik/DLAV-2022', 'size': '6245'}