<jupyter_start><jupyter_text>Stochastic Gradient Descent (SGD) for Linear Regression with L2-regularizationName: <NAME>Unity ID: rspillai  Import Files<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>Prediction Function<jupyter_code>def predict(X, coefficients):
    """
    Input:
        X: Data Row (np.Array)
        coefficients: Coefficients Array (np.Array)
        
    Output:
        Predicted Value for the given Data Row. (float)
    """
    
    #Remove any NAN values from the coefficients to avoid errors.
    coefficients = np.nan_to_num(coefficients)
    
    intercept = coefficients[0]
    predictedValue = 0.0
    
    #Note 1: Skip last value of X as it is the actual Y value.
    #Note 2: Skip coefficients[0] because it is intercept value.
    for i in range(len(X)-1):
        predictedValue += X[i]*coefficients[i+1]
        
    return predictedValue+intercept<jupyter_output><empty_output><jupyter_text>Gradient Descent Function<jupyter_code>def updateCoefficients(X, learningRate, epochs, l2):
    """
    Input:
        X: Data Set (np.Array)
        learningRate: Learning Rate for the gradient descent (float)
        epochs: Number of iterations (int)
        l2: Regularization Constant Lambda (float)
        
    Output:
        coefficients: Array of coefficients (np.Array)
    """
    
    #Initialize the coefficients array to all 0s
    coefficients = np.array([0.0]*len(X[0]))

    #Run for all iterations.
    for epoch in range(epochs):
        
        #Calculate squareError for all iterations.
        squaredError=0
        
        for i in X:      
            predictedValue = predict(i, coefficients)
            error = predictedValue - i[-1]

            squaredError+=error**2
            
            #Update the coefficients.
            coefficients[0]-= learningRate*error
            for j in range(len(i)-1):
                # Updating the coefficients based on L2 Regularization
                coefficients[j+1]-= learningRate*(error*i[j] + l2*coefficients[j+1])
        
        squaredError/=(X.shape[0])
        rms = np.sqrt(squaredError)
        print("Epoch: {}\tRoot Mean Squared Error: {}".format(epoch, squaredError))
        
    return coefficients<jupyter_output><empty_output><jupyter_text>Main Function<jupyter_code>def linearRegression(X_train, learningRate, epochs, l2):
    """
    Input:
        X: Data Set (np.Array)
        learningRate: Learning Rate for the gradient descent (float)
        epochs: Number of iterations (int)
        l2: Regularization Constant Lambda (float)
        
    Output:
        coefficients: Array of coefficients (np.Array)
        
    """
    coefficients = updateCoefficients(X_train, learningRate, epochs, l2)
    return coefficients<jupyter_output><empty_output><jupyter_text>Testing  TrainingTrain the Linear Regression Function using Sklearn's Boston Dataset<jupyter_code>#Import Files
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split

#Load the Boston Dataset
X, Y = load_boston(return_X_y=True)

#Split the dataset into training and testing sets
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.3, random_state=42)

#Normalise the dataset
#Necessary, otherwise Coefficients throw NAN/INF error due to large values.
train_min = X_train.min(0)
train_max = X_train.max(0)
X_train = (X_train - train_min)/(train_max - train_min)
X_test = (X_test - train_min)/(train_max - train_min)

#Combine the Training X & Y set
dataset = np.insert(X_train, X_train.shape[1], values=Y_train, axis=1)

#Calling the Fit function
coefficients = linearRegression(dataset, 0.01, 500, 0.01)

print("\nIntercept:{}\nCoefficients: {}".format(coefficients[0],coefficients[1:]))<jupyter_output>Epoch: 0	Root Mean Squared Error: 106.17661013833626
Epoch: 1	Root Mean Squared Error: 54.647358530589635
Epoch: 2	Root Mean Squared Error: 46.128949505901026
Epoch: 3	Root Mean Squared Error: 41.53308945049576
Epoch: 4	Root Mean Squared Error: 38.39489111772134
Epoch: 5	Root Mean Squared Error: 36.09979694592487
Epoch: 6	Root Mean Squared Error: 34.36280120683172
Epoch: 7	Root Mean Squared Error: 33.016075655193916
Epoch: 8	Root Mean Squared Error: 31.95174358354047
Epoch: 9	Root Mean Squared Error: 31.09711117469757
Epoch: 10	Root Mean Squared Error: 30.401526072909917
Epoch: 11	Root Mean Squared Error: 29.828736194558253
Epoch: 12	Root Mean Squared Error: 29.35220511892027
Epoch: 13	Root Mean Squared Error: 28.952130564747645
Epoch: 14	Root Mean Squared Error: 28.613488372987803
Epoch: 15	Root Mean Squared Error: 28.32471477581022
Epoch: 16	Root Mean Squared Error: 28.076797233391456
Epoch: 17	Root Mean Squared Error: 27.862633558642532
Epoch: 18	Root Mean Squared Error: 27.67657146[...]<jupyter_text>PredictingPredicting the Boston dataset with the built Linear Regression Model<jupyter_code>mseError = 0

#Combining the testing X and Y values
testDataset = np.insert(X_test,X_test.shape[1], values=Y_test, axis=1)

print("Data Sample\tPredicted\tActual\t\tDifference")

#Calling the Prediction function
for i,dataRow in enumerate(testDataset):
    yHat = round(predict(dataRow, coefficients),2)
    print("{}\t\t{}\t\t{}\t\t{}".format(i, yHat, Y_test[i], round(yHat-Y_test[i],2)))
    mseError+= ((yHat-Y_test[i])**2)
    
mseError/=len(X_test)
print("MSE Error: {}".format(mseError))<jupyter_output>Data Sample	Predicted	Actual		Difference
0		27.71		23.6		4.11
1		35.22		32.4		2.82
2		16.17		13.6		2.57
3		25.27		22.8		2.47
4		18.56		16.1		2.46
5		22.89		20.0		2.89
6		18.92		17.8		1.12
7		14.66		14.0		0.66
8		21.31		19.6		1.71
9		21.03		16.8		4.23
10		25.74		21.5		4.24
11		21.08		18.9		2.18
12		-0.16		7.0		-7.16
13		22.25		21.2		1.05
14		20.49		18.5		1.99
15		23.96		29.8		-5.84
16		19.23		18.8		0.43
17		8.12		10.2		-2.08
18		37.46		50.0		-12.54
19		17.76		14.1		3.66
20		26.77		25.2		1.57
21		28.56		29.1		-0.54
22		13.7		12.7		1.0
23		23.56		22.4		1.16
24		17.88		14.2		3.68
25		15.85		13.8		2.05
26		23.21		20.3		2.91
27		15.12		14.9		0.22
28		24.01		21.7		2.31
29		20.22		18.3		1.92
30		22.54		23.1		-0.56
31		25.17		23.8		1.37
32		23.87		15.0		8.87
33		18.59		20.8		-2.21
34		16.09		19.1		-3.01
35		19.5		19.4		0.1
36		30.16		34.7		-4.54
37		21.57		19.5		2.07
38		25.46		24.4		1.06
39		24.79		23.4		1.39
40		15.82		19.7		-3.88
41		29.46		28.2		1.26
42		38.43		50.0		-11.57
43		19.82		17.4	[...]
{'fraction_non_alphanumeric': 0.10393258426966293, 'fraction_numerical': 0.14029338327091137, 'mean_word_length': 3.025753768844221, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '392800', 'license': 'MIT', 'n_tokens_mistral': 3047, 'n_tokens_neox': 2557, 'n_words': 669, 'path': 'rspillai.ipynb', 'repo_name': 'rohanpillai20/SGD-for-Linear-Regression-with-L2-Regularization', 'size': '6414'}