<jupyter_start><jupyter_text>Description and Details:1. Load house sales data2. write a function get_numpy_data to convert into numpy 3. write a function predict_output to compute predictions4. write a function feature_derivative to compute derivative5. Implement gradient descent algorithm from scratch<jupyter_code>import pandas as pd
import numpy as np
import math
print(pd.__version__)<jupyter_output>1.0.5<jupyter_text>Load in house sales data<jupyter_code>sales = pd.read_csv('kc_house_data.csv')
train_data = pd.read_csv('kc_house_train_data.csv')
train_data = pd.read_csv('kc_house_test_data.csv')<jupyter_output><empty_output><jupyter_text>Convert input to numpy array Now we will write a function that will accept an SFrame, a list of feature names (e.g. ['sqft_living', 'bedrooms']) and an target feature e.g. ('price') and will return two things:* A numpy matrix whose columns are the desired features plus a constant column (this is how we create an 'intercept')* A numpy array containing the values of the output<jupyter_code>def get_numpy_data(data, features, output):
    data['constant'] = 1 # add a constant column  
    features = ['constant'] + features # combine two lists

    feature_matrix = data[features].to_numpy()
    output_array = data[output].to_numpy()
    
    return(feature_matrix, output_array)
# test the function:
(example_features, example_output) = get_numpy_data(sales, ['sqft_living'], 'price') 
print (example_features[0,:]) # this accesses the first row of the data the ':' indicates 'all columns'
print (example_output[0]) # and the corresponding output<jupyter_output>[   1 1180]
221900.0<jupyter_text>Predicting output given regression weights<jupyter_code># test:
my_weights = np.array([1., 1.]) # the example weights
my_features = example_features[0,] # we'll use the first data point
predicted_value = np.dot(my_features, my_weights)
print (predicted_value)
def predict_output(feature_matrix, weights):
    predictions = np.dot(feature_matrix, weights)
    return(predictions)
# test:
test_predictions = predict_output(example_features, my_weights)
print (test_predictions[0]) # should be 1181.0
print (test_predictions[1]) # should be 2571.0<jupyter_output>1181.0
2571.0<jupyter_text>Computing the Derivative<jupyter_code>def feature_derivative(errors, feature):
    derivative =  (-2) * np.dot(errors, feature)
    return(derivative)
# test feature_derivative function:

(example_features, example_output) = get_numpy_data(sales, ['sqft_living'], 'price') 
my_weights = np.array([0., 0.]) # this makes all the predictions 0
test_predictions = predict_output(example_features, my_weights) 

errors = example_output - test_predictions
feature = example_features[:,0] 
derivative = feature_derivative(errors, feature) # compute derivative wrt constant
print (derivative)
print (-2 * np.sum(example_output))<jupyter_output>-23345850016.0
-23345850016.0<jupyter_text>Gradient Descent Algorithm<jupyter_code>def regression_gradient_descent(feature_matrix, output, initial_weights, step_size, tolerance):
    converged = False 
    weights = np.array(initial_weights) # make sure it's a numpy array
    while not converged:
        predictions = predict_output(feature_matrix, weights)   
        errors = output - predictions
        gradient_sum_squares = 0 # initialize the gradient sum of squares

        for i in range(len(weights)): 
            derivative = feature_derivative(errors, feature_matrix[:,i])
            weights[i] = weights[i] - step_size * derivative           
            gradient_sum_squares += derivative**2 # get the sum of squared derivatives (for assessing convergence)

        gradient_magnitude = math.sqrt(gradient_sum_squares)                                                                    
        if gradient_magnitude < tolerance:
            converged = True
    return(weights)<jupyter_output><empty_output><jupyter_text>Running the Gradient Descent as Simple Regression<jupyter_code># test the function:
simple_features = ['sqft_living']
my_output = 'price'
(simple_feature_matrix, output) = get_numpy_data(train_data, simple_features, my_output)
initial_weights = np.array([-47000., 1.])
step_size = 7e-12
tolerance = 2.5e7
simple_weights = regression_gradient_descent(simple_feature_matrix, output,initial_weights, step_size,tolerance)
print (simple_weights[1])<jupyter_output>282.3594538991383<jupyter_text>compare with lm in sklearn<jupyter_code>from sklearn import linear_model
lm = linear_model.LinearRegression()
model_1 = lm.fit(train_data['sqft_living'].values.reshape(-1,1), 
                 train_data['price'])
print(model_1.coef_)<jupyter_output>[274.93662162]<jupyter_text>Running a multiple regression<jupyter_code>model_features = ['sqft_living', 'sqft_living15'] # sqft_living15 is the average squarefeet for the nearest 15 neighbors. 
my_output = 'price'
(feature_matrix, output) = get_numpy_data(train_data, model_features, my_output)
initial_weights = np.array([-100000., 1., 1.])
step_size = 4e-12
tolerance = 1e9
multiple_weights = regression_gradient_descent(feature_matrix, output,initial_weights, step_size,tolerance)
multiple_weights<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09438332368268675, 'fraction_numerical': 0.027600849256900213, 'mean_word_length': 5.358282208588957, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '301573', 'license': 'MIT', 'n_tokens_mistral': 1710, 'n_tokens_neox': 1585, 'n_words': 406, 'path': 'week2/HW2_2.ipynb', 'repo_name': 'jessie-sun927/UW-Machine-Learning-Regression', 'size': '5181'}