<jupyter_start><jupyter_text>PIPELINELet's build polynomial regression on the `univariate-2` dataset. using pipelines that organize the various steps of data preprocessing and a training machine learning model, removing tedious steps.  Load and explore the dataperform the usual steps of exploratory data analysis.* load the data, and check for missing values if any. * seperate out the predictor from the response* split the data into the train and test set* visualize the data as a scatter plot<jupyter_code>data = pd.read_csv("../datasets/dataset-2.csv")
data.describe()
data.isnull().sum()
X, y = data[['x']], data['y']
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42, )
plt.scatter(data['x'], data['y'], s=100, alpha=0.6);
plt.xlabel(r'$x\longrightarrow$')
plt.ylabel(r'$y\longrightarrow$')
plt.title(r'Scatterplot visualization of dataset-2 ');<jupyter_output><empty_output><jupyter_text>Standardization of the dataData needs to be scaled, for a variety of reasons. While `sklearn` provides us with a few scalers, by far the most commonly employed is the `StandardScaler`, which z-values the feature.<jupyter_code>scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)<jupyter_output><empty_output><jupyter_text>visualize the effect of scaling the training data. We see that it is now zero-centered $(\mu_x = 0)$. It is not so apparent, but the data scaled to also have unit-variance $(\sigma_x^2 = 1)$.<jupyter_code>plt.scatter(X_train_scaled, y_train, s=100, alpha=0.6);
plt.xlabel(r'$x\longrightarrow$');
plt.ylabel(r'$y\longrightarrow$');
plt.title(r'Scaled predictor: notice how the $x$-values are zero-centered');<jupyter_output><empty_output><jupyter_text>Adding Polynomial FeaturesFrom prior notebook, one can create an effective model by adding polynomial degree terms as predictors.<jupyter_code>poly         = PolynomialFeatures(degree=3)
X_train_poly = poly.fit_transform(X_train_scaled)
X_test_poly  = poly.fit_transform(X_test_scaled)
model = LinearRegression(fit_intercept=True)
model.fit(X_train_poly, y_train)
model.coef_
y_test_hat = model.predict(X_test_poly)
rsquared   = r2_score(y_test_hat, y_test)
print ("The R-squared of the model: {}".format(rsquared))
residuals = y_test_hat - y_test
plt.scatter(y_test_hat, residuals);
plt.xlabel(r'$\hat y\longrightarrow$');
plt.ylabel(r'residuals$\longrightarrow$');
plt.title(r'Residuals plot of the model');<jupyter_output><empty_output><jupyter_text>Creating Data-Processing PipelinesWhile the above appears rather tedious, fortunately sklearn provides us a method to create analytical pipelines. * **Standardize** the input (in this situation, it is not really necessary, but illustrative)   * **Create Polynomial Features** Create polynomials of upto third degree of the input * **Linear Regression** on the resulting standardized, polynomial expansion of the input<jupyter_code>from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
regressor = make_pipeline(StandardScaler(),                     # scale the data
                          PolynomialFeatures(degree=3),         # create polynomial features
                          LinearRegression(fit_intercept=True)) # linear model
regressor.fit(X_train, y_train)
y_test_hat = regressor.predict(X_test)
# The rest of it remains the same
rsquared = r2_score(y_test_hat, y_test)
print ("The R-squared of the model: {}".format(rsquared))<jupyter_output>The R-squared of the model: 0.9670413993139167<jupyter_text>Clean syntax makes for easy interchangeability. For example, we could substitute:* the `StandardScaler` with a `MinMaxScaler`. * substitute the `LinearRegression` with any other regression method, such as `Ridge`, `Lasso`, etc.  Visualization of the model predictionsFinally, let us visualize the model predictions to verify that this exercise still produces the same effective model we have encountered with polynomial regression before.<jupyter_code>xx = np.linspace(0,6,1000)
XX = xx[:, np.newaxis]
yy = regressor.predict(XX)
plt.scatter(data.x, data.y, alpha=0.45)
plt.plot(xx, yy, color="maroon", linewidth=6);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10128388017118402, 'fraction_numerical': 0.011650023775558726, 'mean_word_length': 5.9194078947368425, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '312745', 'license': 'MIT', 'n_tokens_mistral': 1370, 'n_tokens_neox': 1282, 'n_words': 399, 'path': 'notebook/regression-09-pipeline.ipynb', 'repo_name': 'praveenhm/machine-learning-tutorials', 'size': '4206'}