<jupyter_start><jupyter_text>Why NumPyIn the previous class we walked through a bunch of the functionality provided in the NumPy package, but functionality doesn't touch on why or how this package is so useful.In this notebook we will walkthrough some more real world examples on why NumPy is a powerful assest when analyzing datasets in the real world.<jupyter_code>import numpy as np

data = np.array([1,2,3,np.nan])
print(np.nanmean(data))<jupyter_output><empty_output><jupyter_text>Linear RegressionLinear Regression is commonly structured as follows:$$ y = B_01 + B_1x_1 + B_2x_2 + ... + B_nx_n$$Where $B_n$ represents some weight we are applying to some input $x_n$. Another way to phrase this equation is taking the **weighted sum** of our **inputs** to produce a **prediction**.  Setting up our Linear Regression ProblemFirst we need to initialize some data. In our case, we'll use the randomly generated data from earlier.<jupyter_code>import numpy as np
import numpy.random as rand

male_height = rand.normal(loc=70, scale=4, size=1000)
male_weight = rand.normal(loc=170, scale=40, size=1000)
male_age = 3/4 * (male_weight - male_height)
X = np.vstack([male_height, male_weight]).T
y = male_age.reshape(X.shape[0],1)
print(X[:5])
print(y[:5])
%matplotlib inline
import matplotlib.pylab as plt
from mpl_toolkits.mplot3d import Axes3D
    
fig= plt.figure()
axes=fig.add_subplot(111, projection='3d')
axes.scatter(X[:,0], X[:,1], y[0])
plt.show()<jupyter_output><empty_output><jupyter_text>Updating our inputs for an intercept and adding our weightsWe'll actually want to add a placeholder **1** for our inputs for the intercept term. In addition we will want to randomly initialize some weights to go along with our inputs.<jupyter_code>intercept = np.ones(shape=X.shape[0])
int_X = np.vstack([intercept.T, X.T]).T
print(int_X[:5])

weights = rand.normal(size=(3,1))
print(weights)<jupyter_output><empty_output><jupyter_text>Our baseline predictionsBy **weighting** our **inputs** we can get our preliminary **predictions**.<jupyter_code>predictions = int_X.dot(weights)
print(f'Predictions:\n{predictions[:10]}\n')
print(f'Actual:\n{y[:10]}')<jupyter_output><empty_output><jupyter_text>Improving our preliminary predictionsWe'll need some helper functions to help improve these results form their initial values<jupyter_code>def loss(pred, y):
    """Calculates the loss or error in our predictions - MSE
    Args:
        pred (np.array(floats)) - predictions for this iteration
        y (np.array(floats)) - actual
    Returns:
        float - error"""
    return(np.sum(np.power((y-pred), 2))/y.shape[0])

def predict(X, W):
    """Make a predition based on our inputs and weights
    Args:
        X (np.array(floats)) - our inputs
        W (np.array(floats)) - our weights
    Returns:
        np.array(floats) - predictions"""
    
    return(X.dot(W))

def update_weights_linear(X, W, pred, y, lr):
    """Update our weights based on our loss and use of SGD
    Args:
        X -
        W - 
        pred - 
        y - 
        lr (float) - learning rate for our gradient descent
    Returns:
        np.array(float) - updated weights from that cycle"""
    w_gradient = -X.T.dot((y-pred))
    w_gradient /= X.shape[0]
    # This calculates the gradient on a per feature basis
    W = W - (lr * w_gradient).reshape(X.shape[1], 1)
    return(W)<jupyter_output><empty_output><jupyter_text>SGDFor those of you not as familiar with how we train a linear regression model this is a quick recap of SGD or Stochastic Gradient Descent<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.animation as animation


grad = lambda x: 2*x # The derivative, thus gradient, of x^2 is 2x
lr = .05

fig = plt.figure()
ax1 = fig.add_subplot(1, 1, 1)

def animate_sgd(i):
    ax1.clear()
    ax1.plot(x,y)
    ax1.plot(curr_pos[i], curr_pos[i]**2, 'r+')

x = np.arange(-50, 50)
y = np.power(x, 2)
curr_pos = []

curr_x = -50
for _ in range(50):
    curr_x -= grad(curr_x) * lr
    curr_pos.append(curr_x)


anim = animation.FuncAnimation(fig, animate_sgd, frames=range(1, len(curr_pos)), interval=200, repeat=True)
plt.close()
from IPython.display import HTML
HTML(anim.to_jshtml())<jupyter_output><empty_output><jupyter_text>Putting this all together<jupyter_code>W = weights
its = 2000
for ix in range(its):
    preds = predict(int_X, W)
    mse = loss(preds, y)
    if ix % int(its/10) == 0:
        print(f'MSE: {mse}')
    W = update_weights_linear(int_X, W, preds, y, 0.00001)<jupyter_output><empty_output><jupyter_text>Predictions vs Actual<jupyter_code>preds = predict(int_X, W)
print(f'Predictions:\n{preds[:5]}\n')
print(f'Actual:\n{y[:5]}')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12202886247877759, 'fraction_numerical': 0.020161290322580645, 'mean_word_length': 4.448554913294798, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '472001', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 1695, 'n_tokens_neox': 1612, 'n_words': 477, 'path': 'lectures/Linear Regression with Numpy.ipynb', 'repo_name': 'agerner19/agerner19.github.io', 'size': '4712'}