<jupyter_start><jupyter_text>Building two simple polynomial regression model We generate polynomial regressions for two data sets and compare the R2 scores to linear regressions.  Case 1: Profit prediction for an agricultural problemIn the following we would like to predict profits on harvest for certain field sizes.<jupyter_code>#importing the necessary packages
import pandas as pd
import numpy as np

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

from sklearn.preprocessing import PolynomialFeatures
#define data frame
df = pd.read_csv("fields.csv")
df.head()<jupyter_output><empty_output><jupyter_text>We start off with a simple linear regression:<jupyter_code>#define the variables
X = df[["width", "length"]].values
Y = df[["profit"]].values

#split the data set into training and test set
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, random_state = 0, test_size = 0.25)

#train the model
model = LinearRegression()
model.fit(X_train, Y_train)

#report the R2 score
print(model.score(X_test, Y_test))<jupyter_output>0.9265510548118597<jupyter_text>In a next step we proceed with an attempt at polynomial fitting for the data:<jupyter_code>#PolynomialFeatures?

pf = PolynomialFeatures(degree = 2, include_bias = False) #bias term not needed here
#need to fit the training data accordingly (demanded by sklearn) to adapt to polynomial fitting
pf.fit(X_train)

#generate new columns
X_train_transformed = pf.transform(X_train)
X_test_transformed = pf.transform(X_test)

#print all possible arrangements to get to a polynomial of degree 2 (as done by the transform method)
#print(pf.powers_) 

model = LinearRegression()
model.fit(X_train_transformed, Y_train)

print(model.score(X_test_transformed, Y_test))<jupyter_output>0.9878259248140406<jupyter_text>We may redo the analysis without the random_state option in the train test function:<jupyter_code>scores = []
intercepts = []
coefs = []

for i in range(0,1000):
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.25)
    X_train_transformed = pf.transform(X_train)
    X_test_transformed = pf.transform(X_test)

    model = LinearRegression()
    model.fit(X_train_transformed, Y_train)
    
    intercepts.append(model.intercept_)
    coefs.append(model.coef_)
    scores.append(model.score(X_test_transformed, Y_test))
    
print("Average score: " + str(sum(scores)/ len(scores)))

#np.array(coefs).shape<jupyter_output><empty_output><jupyter_text>Now we would like to filter out columns from the fitting procedure:<jupyter_code>scores = []
intercepts = []
coefs = []

for i in range(0,1000):
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.25)
    
    #we may exclude certain columns from the fitting and check if the score improves
    X_train_transformed = pf.transform(X_train)[:, [0, 1, 2, 3, 4]]
    X_test_transformed = pf.transform(X_test)[:, [0, 1, 2, 3, 4]]

    model = LinearRegression()
    model.fit(X_train_transformed, Y_train)
    
    intercepts.append(model.intercept_)
    coefs.append(model.coef_)
    scores.append(model.score(X_test_transformed, Y_test))
    
print("Average score: " + str(sum(scores)/ len(scores)))<jupyter_output>Average score: 0.9868476616439141<jupyter_text>Case 2: Diamond price predictionIn the following we would like to model the prices of diamonds via linear and polynomial regressions and compare the quality of the results via the R2 score.<jupyter_code>#define data frame
df = pd.read_csv("diamonds.csv")
df.head()<jupyter_output><empty_output><jupyter_text>We start off with two simple linear regressions to get a feeling for the system:<jupyter_code># price over carat
#define the variables
X = df[["carat"]].values
Y = df[["price"]].values

#split the data set into training and test set
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, random_state = 0, test_size = 0.25)

#train the model
model = LinearRegression()
model.fit(X_train, Y_train)

#report the R2 score
print(model.score(X_test, Y_test))
# price over dimensions x,y,z
#define the variables
X = df[["x","y","z"]].values
Y = df[["price"]].values

#split the data set into training and test set
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, random_state = 0, test_size = 0.25)

#train the model
model = LinearRegression()
model.fit(X_train, Y_train)

#report the R2 score
print(model.score(X_test, Y_test))<jupyter_output>0.7834062179737853<jupyter_text>Comment: Given this carat seems to be a good indicator for gauging the price of a diamond. Now we try a polynomial regression:<jupyter_code># price over dimensions x,y,z
#define the variables
X = df[["x", "y", "z"]].values
Y = df[["price"]].values

pf = PolynomialFeatures(degree = 2, include_bias = False) #bias term not needed here
#need to fit the training data accordingly (demanded by sklearn) to adapt to polynomial fitting
pf.fit(X_train)

scores = []
intercepts = []
coefs = []

for i in range(0,200):
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.25)
    
    #we may exclude certain columns from the fitting and check if the score improves
    X_train_transformed = pf.transform(X_train)
    X_test_transformed = pf.transform(X_test)
    model = LinearRegression()
    model.fit(X_train_transformed, Y_train)
    
    intercepts.append(model.intercept_)
    coefs.append(model.coef_)
    scores.append(model.score(X_test_transformed, Y_test))
    
print("Average score: " + str(sum(scores)/ len(scores)))<jupyter_output>Average score: -23.75882427600454
{'fraction_non_alphanumeric': 0.10030504216759376, 'fraction_numerical': 0.02494168311501884, 'mean_word_length': 5.451388888888889, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '487436', 'license': 'MIT', 'n_tokens_mistral': 1997, 'n_tokens_neox': 1854, 'n_words': 545, 'path': 'simplepolynomialregressions.ipynb', 'repo_name': 'andreaspts/ML_Linear_Regression_Analyses', 'size': '5573'}