<jupyter_start><jupyter_text>1. Implement a function generating a dataset of $n$ points  according to the following algorithm:  1. Draw $n$ points $x \propto U(0;10)$ (uniformly distributed on $[0,10]$).  2. Draw $n$ points $y \propto \mathcal{N}(1+20x-1.3x^2, 7)$    (from a Gaussian distribution with $\mu=1+20x-1.3x^2$ and $\sigma=7$).  Prepare a dataset of 30 elements and make a scatterplot of  the expected value $y$ in function $x$.2. Use linear regression to fit polynomials to the  generated dataset. Fit polynomials of degrees zero (a constant line),  one, two and three. An easy way to do it is to transform each data  point $x$ into a vector of its powers $[1, x, x^2, \ldots, x^m]$.  Plot the dataset and all regression curves on one figure.  **Note:** The name _linear regression_ signifies that the  hypothesis is linear with respect to parameters $\Theta$.  However, the relationship between $x$ and $y$ is not constrained  to a linear one. In this exercise it is a polynomial one.<jupyter_code>def make_dataset(N):
    X = np.random.uniform(0, 10, N)
    mu = 1 + 20*X - 1.3 * X ** 2
    sigma = 7
    Y = np.random.normal(mu, sigma, N)
    return X, Y

data = make_dataset(30)
plt.scatter(data[0], data[1])
def powers_of_X(X, degree):
    powers = np.arange(degree + 1).reshape(-1,1)
    return X**powers

def compute_polynomial(X, Theta):
    XP = powers_of_X(X, len(Theta) - 1) # len(Theta) x N
    Y = XP * Theta
    Y = np.sum(Y, axis=0)
    return Y

plot_x_space = np.linspace(0,10,100)
plt.scatter(data[0], data[1])
for degree in range(4):
    X = powers_of_X(data[0], degree) # Matrix d x N
    Y = data[1].reshape(1, -1)       # Matrix 1 x N
    Theta = inv(X.dot(X.T)).dot(X).dot(Y.T)
    plt.plot(plot_x_space, compute_polynomial(plot_x_space, Theta).ravel(), 
         label="degree: %d" %(degree, ))
    print(f'Theta:\n{Theta}, Shape: {Theta.shape}\n')
plt.legend(loc='lower right')<jupyter_output>Theta:
[[52.13714049]], Shape: (1, 1)

Theta:
[[16.96227915]
 [ 7.72231598]], Shape: (2, 1)

Theta:
[[-0.12696835]
 [19.3574461 ]
 [-1.1842933 ]], Shape: (3, 1)

Theta:
[[ 4.88150753e-01]
 [ 1.84973532e+01]
 [-9.52420267e-01]
 [-1.60578028e-02]], Shape: (4, 1)
{'fraction_non_alphanumeric': 0.15342465753424658, 'fraction_numerical': 0.07671232876712329, 'mean_word_length': 3.6617021276595745, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '249594', 'license': 'MIT', 'n_tokens_mistral': 926, 'n_tokens_neox': 823, 'n_words': 264, 'path': 'Projects/REGRESSION/.ipynb_checkpoints/linear_regression-checkpoint.ipynb', 'repo_name': 'TheFebrin/Machine-Learning', 'size': '2190'}