<jupyter_start><jupyter_text>Inverse Linear Regression Proof In this notebook we want to check if both ways of 'inverse linear regression' lead to the same result! Write the function that uses expenses as input and sqft_living as output!* Upload and preprocess the data* Write a function to compute the Simple Linear Regression weights with sqft_living as output and price as input* Write a function to make predictions of the output given the input feature of $1.000.000 Then write a function that uses sqft_living as input and price as output and invert it!* Write a function to compute the Simple Linear Regression weights with price as output and sqft_living as input* Write a function to make predictions of the input given the output feature of $1.000.000  Upload and preprocess the dataDataset on house sales data (housing.csv).<jupyter_code>import pandas as pd
data = pd.read_csv('housing.csv')
# Look at the table to check potential features
data[:10]<jupyter_output><empty_output><jupyter_text>Split data into training and testing<jupyter_code>from sklearn.model_selection import train_test_split
# Split data set into 80% train and 20% test data 
train_data, test_data = train_test_split(data, test_size=0.2, random_state=0)
# Look at the shape to check the split ratio
train_data.shape, test_data.shape<jupyter_output><empty_output><jupyter_text>Build a generic simple linear regression function  Let's build a closed form solution (function) to compute the slope and intercept for a simple linear regression. Sample regression:\begin{equation}\hat{y}_i = \hat{i} + \hat{s} x_i\end{equation}Sample intercept:\begin{equation}\hat{i} = \bar{y} – \hat{s} \bar{x}\end{equation}Sample slope:\begin{equation}\hat{s} = \frac{\sum(x_i – \bar{x}) (y_i – \bar{y})} {\sum(x_i – \bar{x})^2}\end{equation}Here are the sample means:\begin{equation}\bar{x} = \frac{1}{n}\sum_{i=1}^n x_i\end{equation}\begin{equation}\bar{y} = \frac{1}{n}\sum_{i=1}^n y_i\end{equation}<jupyter_code>from time import time

X = train_data['price'].values
Y = train_data['sqft_living'].values<jupyter_output><empty_output><jupyter_text>What a Statistician would do:<jupyter_code>def simple_linear_regression(X, Y):
    
    # Compute the means:
    assert len(X) == len(Y)
    
    n = len(X)
    x_mean = sum(X) / n
    y_mean = sum(Y) / n
    
    # Compute the slope:
    slope = sum((X - x_mean) * (Y - y_mean)) / sum((X - x_mean)**2)
    
    # Compute the intercept:
    intercept = y_mean - slope * x_mean
    
    return (intercept, slope)
t0 = time()
sqft_intercept, sqft_slope = simple_linear_regression(X, Y)
t1 = time()

print('Elapsed time: {:.2f} milliseconds\n'.format((t1 - t0) * 1000))
print('Intercept: ' + str(sqft_intercept))
print('Slope: ' + str(sqft_slope))<jupyter_output>Elapsed time: 5.48 milliseconds

Intercept: 1134.23538705004
Slope: 0.0017449919139801982<jupyter_text>What a Machine Learning Engineer would do:<jupyter_code>from sklearn.linear_model import LinearRegression

def simple_linear_regression_sklearn(X, Y):
    X_2D = X.reshape(-1, 1)  # Convert to 2D array for Sklearn
    model = LinearRegression().fit(X_2D, Y)
    return (model.intercept_, model.coef_[0])
t0 = time()
sqft_intercept, sqft_slope = simple_linear_regression_sklearn(X, Y)
t1 = time()

print('Elapsed time: {:.2f} milliseconds\n'.format((t1 - t0) * 1000))
print('Intercept: ' + str(sqft_intercept))
print('Slope: ' + str(sqft_slope))<jupyter_output>Elapsed time: 1.36 milliseconds

Intercept: 1134.235387050036
Slope: 0.0017449919139802053<jupyter_text>What a Computer Scientist would do:<jupyter_code>def simple_linear_regression_optimized(input_feature, output):
    
    n = len(input_feature)
    
    # compute the sum of input_feature and output
    x_sum = input_feature.sum()
    y_sum = output.sum()
    
    # compute the product of the output and the input_feature and its sum
    xy = input_feature * output
    xy_sum = xy.sum()
    
    # compute the squared value of the input_feature and its sum
    x_squared = input_feature * input_feature
    x_squared_sum = x_squared.sum()
    
    # use the formula for the slope
    slope = (xy_sum - x_sum * y_sum / n) / (x_squared_sum - x_sum * x_sum / n)
    
    # use the formula for the intercept
    intercept = y_sum / n - slope * x_sum / n
    
    return (intercept, slope)
t0 = time()
sqft_intercept, sqft_slope = simple_linear_regression_optimized(X, Y)
t1 = time()

print('Elapsed time: {:.2f} milliseconds\n'.format((t1 - t0) * 1000))
print('Intercept: ' + str(sqft_intercept))
print('Slope: ' + str(sqft_slope))<jupyter_output>Elapsed time: 0.35 milliseconds

Intercept: 1134.235387050036
Slope: 0.001744991913980205<jupyter_text>Plot data and function  Predicting values Now let's use that function of a simple regression model for predicting prices based on sqft - rembember that we train on train_data!With the model parameters 'intercept & slope' we can now write a function to return the predicted output - given the input_feature, slope and intercept:<jupyter_code>def get_regression_predictions(input_feature, intercept, slope):
    # calculate the predicted values:
    predicted_values = intercept + slope * input_feature
    return predicted_values
import matplotlib.pyplot as plt
import numpy as np

def plot_function(intercept, slope):
    axes = plt.gca() # get current axes
    x_vals = np.array(axes.get_xlim())
    y_vals = get_regression_predictions(x_vals, intercept, slope)
    plt.plot(x_vals, y_vals, 'r-')
# Plot feature relations
plt.figure(figsize=(5,10))

plt.plot(X, Y, 'o')
plt.title('price vs. sqft_living')
plt.xlabel('sqft_living')
plt.ylabel('price')

plot_function(sqft_intercept, sqft_slope)<jupyter_output><empty_output><jupyter_text>Additional Task: Go to your 2a insurance notebook from yesterday and fit a regression line into your plot of bmi vs. expenses Now that we have this function let's make a prediction. * What is the estimated expense for an entry according to the model we estiamted above?<jupyter_code>import locale
locale.setlocale(locale.LC_ALL, 'en_US')

my_budget = 1000000
estimated_sqft_living = get_regression_predictions(my_budget, sqft_intercept, sqft_slope)

my_budget_currency = locale.currency(my_budget, grouping=True)
print("The estimated sqft_living for a budget of {} is {:.2f}".format(my_budget_currency, estimated_sqft_living))<jupyter_output>The estimated sqft_living for a budget of $1,000,000.00 is 2879.23<jupyter_text>Inverse Regression PredictionThe inverse process is calculated by solving `output = intercept + slope * input_feature` for `input_feature`.<jupyter_code>def inverse_regression_predictions(output, intercept, slope):
    # Use this equation to compute the inverse predictions:
    estimated_feature = (output - intercept) / slope
    return estimated_feature
my_sqft = estimated_sqft_living
estimated_budget = inverse_regression_predictions(my_sqft, sqft_intercept, sqft_slope)

estimated_budget_currency = locale.currency(my_budget, grouping=True)
print("The estimated budget for a living of {:.2f} sqft is {}".format(estimated_sqft_living, estimated_budget_currency))<jupyter_output>The estimated budget for a living of 2879.23 sqft is $1,000,000.00<jupyter_text>Train an inverse model that predicts the budget based on the sqft_livingroom:<jupyter_code>budget_intercept, budget_slope = simple_linear_regression(Y, X)

print('Intercept: ' + str(budget_intercept))
print('Slope: ' + str(budget_slope))
# Plot feature relations
plt.figure(figsize=(10,5))

plt.plot(Y, X, 'o')
plt.title('price vs. sqft_living')
plt.xlabel('sqft_living')
plt.ylabel('price')

plot_function(budget_intercept, budget_slope)<jupyter_output><empty_output><jupyter_text>Compare the output of the inverse model with an inverse prediction, against the first model:<jupyter_code>my_budget_2 = 1000000
estimated_sqft_living_2 = inverse_regression_predictions(my_budget_2, budget_intercept, budget_slope)

my_budget_2_currency = locale.currency(my_budget_2, grouping=True)
print("The estimated sqft_living for a budget of {} is {:.2f}".format(my_budget_2_currency, estimated_sqft_living_2))<jupyter_output>The estimated sqft_living for a budget of $1,000,000.00 is 3691.45<jupyter_text>The short answer is [here](https://stats.stackexchange.com/a/20560):<jupyter_code>plt.figure(figsize=(15,10))
plt.imshow(plt.imread('regression.png'))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10067994751282357, 'fraction_numerical': 0.03125372778241679, 'mean_word_length': 5.529595015576324, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 43, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '596268', 'license': 'MIT', 'n_tokens_mistral': 3038, 'n_tokens_neox': 2790, 'n_words': 781, 'path': 'Data-Analysis/5_Linear-regression/3_inverse-linear-regression_housing_solution.ipynb', 'repo_name': 'Heldenkombinat/Tutorials', 'size': '8383'}