<jupyter_start><jupyter_text>Simple PipelinesIn the previous chapters on transformations, we took our original data, imputed missing values, and then scaled it. We applied two successive transformations in a rather manual and clumsy fashion by using intermediate variables to store the results of the first transformation.In this chapter, we learn about the scikit-learn Pipeline meta-estimator which is able to apply any number of transformations in succession. This Pipeline object simplifies the process, makes it more robust, less error-prone, and allows the transformers to be part of a grid search. Successive transformations without a PipelineLet's begin by reading in our data and using transformers to impute missing values and standardize the input data in the same fashion we did in the previous chapters without using the Pipeline.<jupyter_code>import pandas as pd
housing = pd.read_csv('../data/housing_sample.csv')
X = housing[['GrLivArea', 'GarageArea', 'LotFrontage', 'OverallQual']]
y = housing['SalePrice']
X.head()<jupyter_output><empty_output><jupyter_text>We filled in missing values with the mean and assigned the result to a new variable, `X_filled` in this case.<jupyter_code>from sklearn.impute import SimpleImputer
si = SimpleImputer(strategy='mean')
X_filled = si.fit_transform(X)<jupyter_output><empty_output><jupyter_text>We then took this result and standardized it with the `StandardScaler` transformer.<jupyter_code>from sklearn.preprocessing import StandardScaler
ss = StandardScaler()
X_scaled = ss.fit_transform(X_filled)<jupyter_output><empty_output><jupyter_text>Automating transformations with the PipelineWhile the above workflow is valid, it's a bit clumsy as we must keep track of each transformed dataset. If we had several more transformations, then we would be creating many intermediate variables. scikit-learn provides us a nicer workflow with the `Pipeline` meta-estimator. This pipeline applies one transformation after the other without the need to use intermediate variables. You just need to create a list of the steps it takes. Create steps - a list of two-item tuples - name, estimatorIn order to use the scikit-learn `Pipeline`, you must create the steps it will take. Each step is a transformation you'd like applied to your dataset. The steps are created as a list of two-item tuples. The first item of the tuple is the **name** of the step as a string. The name is arbitrary and can be anything. The second item of the tuple is an instantiated **estimator** that dictates what will happen. The estimator must be a transformer unless it's the last step, in which it can be either a transformer or machine learning model.The `Pipeline` estimator is found in the `pipeline` module. Let's import it and create our list of steps. We'll compute the same two transformations on our data as above. The transformers are instantiated again as to not use any information from the previous result.<jupyter_code>from sklearn.pipeline import Pipeline
si = SimpleImputer(strategy='mean')
ss = StandardScaler()
steps = [('si', si), ('ss', ss)]<jupyter_output><empty_output><jupyter_text>This pipeline consists of two steps - imputation and standardization. I've chosen the strings 'si' and 'ss' for the names of each step, but you are free to name them however you wish. The scikit-learn pipeline follows the same three-step process like any other estimator. We've only imported it at this time. Let's instantiate it by passing it the steps we created.<jupyter_code>pipe = Pipeline(steps)<jupyter_output><empty_output><jupyter_text>We can now fit this pipeline with our original data. This particular pipeline consists of only transformers and no machine learning estimators (we'll soon see a pipeline that does have a machine learning estimator as the final step). We can fit it by just passing in our input data.<jupyter_code>pipe.fit(X)<jupyter_output><empty_output><jupyter_text>As always, the `fit` method for transformers learns how to transform the data, but does not return any transformed data. To return a transformed dataset, call the `trnsform` method. Missing values get imputed first and then all of the data is standardized. The first three rows of the transformed data are output.<jupyter_code>X_t = pipe.transform(X)
X_t[:3].round(2)<jupyter_output><empty_output><jupyter_text>The `fit_transform` method is available to learn from the data and transform it simultaneously. It produces the same result as above.<jupyter_code>X_t = pipe.fit_transform(X)
X_t[:3].round(2)<jupyter_output><empty_output><jupyter_text>Machine learning pipelinesAfter transforming the data, we are ready to do machine learning with it. We could use the variable `X_t` from above and pass it to the `fit` method of a machine learning estimator, but there is a better way. Luckily for us, scikit-learn pipelines allow for machine learning models to be the last step. Below, we instantiate a k-nearest neighbors estimator and add it as the last step in our pipeline. Finally, we instantiate a new pipeline and assign it to the variable name `pipe_ml`.<jupyter_code>from sklearn.neighbors import KNeighborsRegressor
si = SimpleImputer(strategy='mean')
ss = StandardScaler()
knr = KNeighborsRegressor()
steps = [('si', si), ('ss', ss), ('knr', knr)]
pipe_ml = Pipeline(steps)<jupyter_output><empty_output><jupyter_text>This pipeline imputes missing values, standardizes the data, and then learns using k-nearest neighbors on that transformed data. Each step in a pipeline must be a transformer, with the last step being an exception. It can be either a transformer or a machine learning estimator. No other steps can follow a machine learning estimator. If the last step is a machine learning estimator, as is the case here, then you must pass in both the input (`X`) and output (`y`) variables. ![][1]You cannot call `fit_transform` or `fit` and then `transform` as machine learning estimators do not transform the data. If you attempt to do so, you'll get the following error.[1]: images/basic_pipeline.png<jupyter_code>pipe_ml.fit_transform(X, y)<jupyter_output><empty_output><jupyter_text>Instead, you must first call the `fit` method, and then use other methods such as `predict` or `score`. Let's begin by calling the `fit` method on our machine learning pipeline.<jupyter_code>pipe_ml.fit(X, y);<jupyter_output><empty_output><jupyter_text>It might be helpful to reconstruct this sequence without using a pipeline to compare how things would look if this pipeline object did not exist in scikit-learn.<jupyter_code>X_filled = si.fit_transform(X)
X_scaled = ss.fit_transform(X_filled)
knr.fit(X_scaled, y);<jupyter_output><empty_output><jupyter_text>This machine learning pipeline has learned how to impute the data with the given strategy, standardize the data, and build a k-nearest neighbors model with this imputed and standardized data. We can now use it to make predictions with the `predict` method. When making the call to `predict`, it passes the input through each step of the pipeline. Here, we make predictions on the input data that we trained the model with.<jupyter_code>pipe_ml.predict(X)<jupyter_output><empty_output><jupyter_text>The `score` method is available to evaluate performance and uses the same `score` method as the machine learning estimator provided in the last step. It also passes the data through each step of the pipeline.<jupyter_code>pipe_ml.score(X, y)<jupyter_output><empty_output><jupyter_text>Using cross validation on a pipelineAs we know well by now, our result from the last call to the `score` method doesn't properly evaluate our model. Machine learning pipelines can be cross-validated just like any other supervised machine learning estimator. Let's compute the mean score of a 5-fold shuffled cross validation.<jupyter_code>from sklearn.model_selection import KFold, cross_val_score
kf = KFold(n_splits=5, shuffle=True, random_state=123)
cross_val_score(pipe_ml, X, y, cv=kf).mean()<jupyter_output><empty_output><jupyter_text>Any machine learning estimator can end the pipelineAll machine learning estimators may be used at the end of a pipeline. Let's use a random forest with 50 trees and a `max_depth` set to 5. Note, that we removed the standardization step because it does not change how decision trees are created.<jupyter_code>from sklearn.ensemble import RandomForestRegressor
rfr = RandomForestRegressor(n_estimators=50, max_depth=5)
steps = [('si', si), ('rfr', rfr)]
pipe_ml2 = Pipeline(steps)
cross_val_score(pipe_ml2, X, y, cv=kf).mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06633964027096473, 'fraction_numerical': 0.002219107685120299, 'mean_word_length': 5.8504, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 65, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '568824', 'license': 'MIT', 'n_tokens_mistral': 2398, 'n_tokens_neox': 2295, 'n_words': 1109, 'path': 'jupyter_notebooks/machine_learning/mastering_machine_learning/06. Data Transformations/03. Simple Pipelines.ipynb', 'repo_name': 'manual123/Nacho-Jupyter-Notebooks', 'size': '8562'}