<jupyter_start><jupyter_text>This post addresses exercise 2 of chapter 1 from the book The Essence of Neural Networks. In this exercise, we fit a line to the given data using linear least squares. The method of linear least squares provides us a way of calculating slop and intercept. A straight line with this slop and intercept minimizes the sum of the squared errors for all data points.<jupyter_code>"""
Ex2: In this exercise one needs to fit a straight line to the
data already provided in Question 1 (i.e., ex1) using least squares.

The linear least squares fitting technique is the simplest and 
most commonly applied form of linear regression and provides a 
solution to the problem of finding the best fitting straight line 
through a set of points.

Method of least squares provides us a way of calculating slop m
and intercept c. The straight line with this slop m and intercept 
c minimizes the sum of the squared errors for all data points.
The sum of squared error is simply these errors squared and summed.

The method of least squares tells us that m and c are found from:
m = (n*sum(x_i*y_i) - sum(x_i)*sum(y_i)) / (n*sum(x_i**2) - (sum(x_i))**2)

c = sum(y_i) - m*sum(x_i)

Here, x_i and y_i are the coordinate values for point i and n is the
number of points. The summations are performed for all data points.
"""


import numpy as np
import matplotlib.pyplot as plt
#from sklearn.metrics import mean_squared_error

# input data points, x-coordinates
x = np.array([0.30, 0.35, 0.40, 0.50, 0.60, 0.80, 0.95, 1.10])
# target output or true output, i.e., true y-coordinates
target = np.array([1.60, 1.40, 1.40, 1.60, 1.70, 2.00, 1.70, 2.10])

# number of data points
n = len(x)

# slop using least square method
m = (n*np.sum(x*target) - np.sum(x)*np.sum(target)) / (n*np.sum(x**2) - (np.sum(x))**2)
# intercept using least square method
c = (np.sum(target) - m*np.sum(x)) / n

# using above calculated m and c, fit a line on data points x.
y = m*x + c

plt.ion()
fig = plt.figure()
p1 = fig.add_subplot(111)
p1.plot(x, y, label='Least square fit')
p1.plot(x, target, 'bo', label='data points')
plt.legend()
p1.set_title("Least square fit to data points")
p1.set_xlabel("X Axis")
p1.set_ylabel("Y Axis")
#fig.savefig('Ex2.png', dpi=300)

# mean squared error after fitting line 1 (i.e., values of output1) and target output
mse1 = np.mean((target - y)**2)
print('Mean squared error after least square fit: {}'.format(mse1))

# mean squared error using sklearn
#print(mean_squared_error(target, y))<jupyter_output>Mean squared error after least square fit: 0.016871157786885263
{'fraction_non_alphanumeric': 0.10451214809101426, 'fraction_numerical': 0.036251446201311224, 'mean_word_length': 4.283095723014257, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '425476', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 945, 'n_tokens_neox': 854, 'n_words': 371, 'path': 'the-essence-of-neural-networks/exercise-2.ipynb', 'repo_name': 'vinaymodi/notebooks', 'size': '2593'}