<jupyter_start><jupyter_text>CrossColumnAddTransformerThis notebook shows the functionality in the CrossColumnAddTransformer class. This transformer changes the values of one column via an additive adjustment, based on the values in other columns.<jupyter_code>import pandas as pd
import numpy as np
import tubular
from tubular.mapping import CrossColumnAddTransformer
tubular.__version__<jupyter_output><empty_output><jupyter_text>Load Boston house price dataset from sklearnNote, the load_boston script modifies the original Boston dataset to include nulls values and pandas categorical dtypes.<jupyter_code>boston_df = tubular.testing.test_data.prepare_boston_df()
boston_df.shape
boston_df.head()
boston_df.dtypes<jupyter_output><empty_output><jupyter_text>Simple usage  Initialising CrossColumnAddTransformer The user must pass in a dict of mappings, each item within must be a dict of mappings for a specific column. The column to be adjusted is also specified by the user. As shown below, if not all values of a column are required to define mappings, then these can be excluded from the dictionary. All additive adjustments defined must be numeric (int or float)<jupyter_code>mappings = {
    'RAD': {
        '1.0': 1.1,
        '2.0': 0.5,
        '3.0': -4,        
    }
}

adjust_column = "target"
map_1 = CrossColumnAddTransformer(adjust_column = adjust_column, mappings = mappings, copy = True, verbose = True)<jupyter_output>BaseTransformer.__init__() called<jupyter_text>CrossColumnAddTransformer fitThere is not fit method for the CrossColumnAddTransformer as the user sets the mappings dictionary when initialising the object.  CrossColumnAddTransformer transformOnly one column mapping was specified when creating map_1 so only this column will be all be used to adjust the value of the adjust_column when the transform method is run.<jupyter_code>boston_df[['RAD','target']].head(10)
boston_df[boston_df['RAD'].isin(['1.0', '2.0','3.0'])]['target'].groupby(boston_df['RAD']).mean()
boston_df_2 = map_1.transform(boston_df)
boston_df_2[['RAD','target']].head(10)
boston_df_2[boston_df_2['RAD'].isin(['1.0', '2.0','3.0'])]['target'].groupby(boston_df_2['RAD']).mean()<jupyter_output><empty_output><jupyter_text>Column dtype conversionIf all the column to be adjusted has dtype int, but the additive adjustments specified are non-integer, then the column will be converted to a float dtype.<jupyter_code>mappings_2 = {
    'RAD': {
        '1.0': 1.1,
        '2.0': 0.5,
        '3.0': 4,        
    }
}

adjust_column_2 = "CHAS"
map_2 = CrossColumnAddTransformer(adjust_column = adjust_column_2, mappings=mappings_2, copy = True, verbose = True)
boston_df_3 = boston_df_2.copy()
boston_df_3['CHAS'] = boston_df_3['CHAS'].astype(float).astype(int)
boston_df_3['CHAS'].dtype
boston_df_3['CHAS'].value_counts(dropna = False)
boston_df_4 = map_2.transform(boston_df_3)
boston_df_4['CHAS'].dtype
boston_df_4['CHAS'].value_counts(dropna = False)<jupyter_output><empty_output><jupyter_text>Specifying multiple columns If more than one column is used to define the mappings, then as addition is a commutative operation it does not matter which order the multipliers are applied in.<jupyter_code>mappings_4 = {
    'RAD': {
        '1.0': 1.1,
        '2.0': 0.5,
        '3.0': -4,        
    },
    'ZN': {
        '0.0': -3,
    }
}

adjust_column_4 = "target"
map_4 = CrossColumnAddTransformer(adjust_column = adjust_column_4, mappings = mappings_4, copy = True, verbose = True)
boston_df[['ZN','RAD','target']].head()<jupyter_output><empty_output><jupyter_text>In the above example, the target would only be adjusted for row 1 by adding 0.5 (as RAD=2.0), whereas row 2 would be adjusted by subtracting 2.5 (RAD=2.0 means adding 0.5, ZN=0.0 means subtracting 3, 0.5-3=-2.5)<jupyter_code>boston_df_5 = map_4.transform(boston_df)
boston_df_5[['ZN','RAD','target']].head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10878447395301327, 'fraction_numerical': 0.026557711950970377, 'mean_word_length': 5.3279483037156705, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '180917', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1345, 'n_tokens_neox': 1276, 'n_words': 361, 'path': 'examples/mapping/CrossColumnAddTransformer.ipynb', 'repo_name': 'munichpavel/tubular', 'size': '3916'}