<jupyter_start><jupyter_text>ScalingTransformerThis notebook shows the functionality of the `ScalingTransformer` class. This transformer allows the user to perform different types of scaling to data, `X`. There are 3 types of scaling from the `sklearn.preprocessing` module that this transformer allows users to access;- min max scaling from the [MinMaxScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.htmlsklearn.preprocessing.MinMaxScaler)- max absolute scaling from the [MaxAbsScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MaxAbsScaler.htmlsklearn.preprocessing.MaxAbsScaler)- standardisation from the [StandardScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.htmlsklearn.preprocessing.StandardScaler)<jupyter_code>import pandas as pd
import numpy as np
import tubular
from tubular.numeric import ScalingTransformer
tubular.__version__<jupyter_output><empty_output><jupyter_text>Load Boston house price dataset from sklearnNote, the load_boston script modifies the original Boston dataset to include nulls values and pandas categorical dtypes.<jupyter_code>boston_df = tubular.testing.test_data.prepare_boston_df()
boston_df.shape
boston_df.head()
boston_df.dtypes<jupyter_output><empty_output><jupyter_text>Simple usage  Initialising ScalingTransformer When initialising the `ScalingTransformer` the user must specify;- `columns` the columns to apply scaling to- `scaler` which must be one of 'min_max', 'max_abs', 'standard', this argument is used to set which type of scaling will be usedThe user can also specify `scaler_kwargs` which is a dictionary of keyword arguments that will be passed to the scaler when it is initalised. See the docs for each scaler ([MinMaxScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.htmlsklearn.preprocessing.MinMaxScaler), [MaxAbsScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MaxAbsScaler.htmlsklearn.preprocessing.MaxAbsScaler), [StandardScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.htmlsklearn.preprocessing.StandardScaler)) for the different arguments that each accepts.<jupyter_code>scaling_1 = ScalingTransformer(columns = ['NOX', 'CRIM'], scaler = 'standard')<jupyter_output><empty_output><jupyter_text>ScalingTransformer fitThe `ScalingTransformer` must be `fit` on data before running `transform` to apply the scaling. The `fit` method determines the scaling values based off the input data.<jupyter_code>scaling_1.fit(boston_df)<jupyter_output>C:\Anaconda\envs\python3\lib\site-packages\sklearn\base.py:197: FutureWarning: From version 0.24, get_params will raise an AttributeError if a parameter cannot be retrieved as an instance attribute. Previously it would return None.
  FutureWarning)<jupyter_text>ScalingTransformer transformThe `transform` method applies scaling to the input data `X`. The specified columns are modified in place rather than creating new columns.<jupyter_code>boston_df['NOX'].quantile([0, 0.2, 0.4, 0.6, 0.8, 1.0])
boston_df['CRIM'].quantile([0, 0.2, 0.4, 0.6, 0.8, 1.0])
boston_df_2 = scaling_1.transform(boston_df)
boston_df_2['NOX'].quantile([0, 0.2, 0.4, 0.6, 0.8, 1.0])
boston_df_2['CRIM'].quantile([0, 0.2, 0.4, 0.6, 0.8, 1.0])<jupyter_output><empty_output><jupyter_text>Max absolute scalingBelow is an example of using max absolute scaling.<jupyter_code>scaling_2 = ScalingTransformer(columns = ['NOX', 'CRIM'], scaler = 'max_abs')
scaling_2.fit(boston_df)
boston_df_3 = scaling_2.transform(boston_df)
boston_df_3['NOX'].quantile([0, 0.2, 0.4, 0.6, 0.8, 1.0])
boston_df_3['CRIM'].quantile([0, 0.2, 0.4, 0.6, 0.8, 1.0])<jupyter_output><empty_output><jupyter_text>Min max scalingBelow is an example of using min max scaling.<jupyter_code>scaling_3 = ScalingTransformer(columns = ['NOX', 'CRIM'], scaler = 'min_max')
scaling_3.fit(boston_df)
boston_df_4 = scaling_3.transform(boston_df)
boston_df_4['NOX'].quantile([0, 0.2, 0.4, 0.6, 0.8, 1.0])
boston_df_4['CRIM'].quantile([0, 0.2, 0.4, 0.6, 0.8, 1.0])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11353816610657705, 'fraction_numerical': 0.027364378300528083, 'mean_word_length': 6.9674952198852775, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '650962', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1438, 'n_tokens_neox': 1368, 'n_words': 339, 'path': 'examples/numeric/ScalingTransformer.ipynb', 'repo_name': 'munichpavel/tubular', 'size': '4166'}