<jupyter_start><jupyter_text>Problem 4, Parts F-H: Stochastic Gradient Descent with a Larger Dataset Use this notebook to write your code for problem 4 parts F-H by filling in the sections marked ` TODO` and running all cells.<jupyter_code># Setup.

import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Problem 4F: Perform SGD with the new dataset For the functions below, you may re-use your code from parts 4C-E. Note that you can now modify your SGD function to return the final weight vector instead of the weights after every epoch.<jupyter_code>def loss(X, Y, w):
    '''
    Calculate the squared loss function.
    
    Inputs:
        X: A (N, D) shaped numpy array containing the data points.
        Y: A (N, ) shaped numpy array containing the (float) labels of the data points.
        w: A (D, ) shaped numpy array containing the weight vector.
    
    Outputs:
        The loss evaluated with respect to X, Y, and w.
    '''
    
    #==============================================
    # TODO: Implement the SGD loss function.
    #==============================================
    
    pass

def gradient(x, y, w):
    '''
    Calculate the gradient of the loss function with respect to
    a single point (x, y), and using weight vector w.
    
    Inputs:
        x: A (D, ) shaped numpy array containing a single data point.
        y: The float label for the data point.
        w: A (D, ) shaped numpy array containing the weight vector.
        
    Output:
        The gradient of the loss with respect to x, y, and w. 
    '''
    
    #==============================================
    # TODO: Implement the gradient of the loss function.
    #==============================================    
    
    pass

def SGD(X, Y, w_start, eta, N_epochs):
    '''
    Perform SGD using dataset (X, Y), initial weight vector w_start,
    learning rate eta, and N_epochs epochs.
    
    Inputs:
        X: A (N, D) shaped numpy array containing the data points.
        Y: A (N, ) shaped numpy array containing the (float) labels of the data points.
        w_start:  A (D, ) shaped numpy array containing the weight vector initialization.
        eta: The step size.
        N_epochs: The number of epochs (iterations) to run SGD.
        
    Outputs:
        w: A (D, ) shaped array containing the final weight vector.
        losses: A (N_epochs, ) shaped array containing the losses from all iterations.
    '''
    
    #==============================================
    # TODO: Implement the SGD algorithm.
    #==============================================    
    
    pass<jupyter_output><empty_output><jupyter_text>Next, we need to load the dataset. In doing so, the following function may be helpful:<jupyter_code>def load_data(filename):
    """
    Function loads data stored in the file filename and returns it as a numpy ndarray.
    
    Inputs:
        filename: GeneratorExitiven as a string.
    
    Outputs:
        Data contained in the file, returned as a numpy ndarray
    """
    return np.loadtxt(filename, skiprows=1, delimiter=',')<jupyter_output><empty_output><jupyter_text>Now, load the dataset in `sgd_data.csv` and run SGD using the given parameters; print out the final weights.<jupyter_code>#==============================================
# TODO:
# (1) load the dataset
# (2) run SGD using the given parameters
# (3) print out the final weights.
#==============================================

# The following should help you get started:
data = load_data('https://raw.githubusercontent.com/lakigigar/Caltech-CS155-2021/main/psets/set1/data/sgd_data.csv')<jupyter_output><empty_output><jupyter_text>Problem 4G: Convergence of SGD This problem examines the convergence of SGD for different learning rates. Please implement your code in the cell below:<jupyter_code>#==============================================
# TODO: create a plot showing the convergence
# of SGD for the different learning rates.
#==============================================<jupyter_output><empty_output><jupyter_text>Problem 4H Provide your code for computing the least-squares analytical solution below.<jupyter_code>#==============================================
# TODO: implement the least-squares
# analytical solution.
#==============================================<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.19222020018198363, 'fraction_numerical': 0.004094631483166515, 'mean_word_length': 3.6826411075612353, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '490436', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 1218, 'n_tokens_neox': 1171, 'n_words': 476, 'path': 'psets/set1/set1_prob4_part2.ipynb', 'repo_name': 'ichakraborty/CS155-iniproject', 'size': '4396'}