<jupyter_start><jupyter_text>If you are getting an error on importing ColumnTransformer, it might be because you might be using an old version of `scikit-learn`. To update `scikit-learn` to the latest version, type the following into the terminal or command prompt. ```pip install -U scikit-learn``` Then hit enter.  Here, I am constructing a simple data frame for this tutorial.<jupyter_code>d = {'col_1': np.linspace(0, 1.5, 5),
    'col_2': ['A', 'B', 'C', 'D', 'E'],
    'col_3': ['f', 'g', 'h', 'i', 'j'],
    'col_4': [5, 6, 7, 8, 9]}
pd.DataFrame(d)
data = pd.DataFrame(d)<jupyter_output><empty_output><jupyter_text>Let us have a look at our data frame.<jupyter_code>data<jupyter_output><empty_output><jupyter_text>`col_2` and `col_3` are categorical features.  It is recommended to preprocess categorical features with One hot encoding, before passing it to the machine learning model. Let us just preprocess one feature first. **`ColumnTransformer`** takes in a list of tuples; Each tuple will have (name,  transformer, column_name(s)) For this example, I **named** the tuple as ‘categorical_features’; you can call it anything; it does not affect the process. **Transformer** here is OneHotEncoder from scikit-learn; Note that the transformer must support fit and transform methods, for example, we cannot use get_dummies from Pandas here. **Column name** here is ‘col_2’, we will use multiple columns in the next example<jupyter_code>from sklearn.preprocessing import OneHotEncoder
ColumnTransformer([('categorical_features', OneHotEncoder(sparse=False), ['col_2'])])<jupyter_output><empty_output><jupyter_text>Saving it in a variable ‘ct’.<jupyter_code>ct = ColumnTransformer([('categorical_features', OneHotEncoder(sparse=False, drop = 'first'), ['col_2'])])<jupyter_output><empty_output><jupyter_text>Here the first column indicates ‘B’ of ‘col_2’, second to ‘C’ and so forth.<jupyter_code>ct.fit_transform(data)
data<jupyter_output><empty_output><jupyter_text>Let us take a look at multiple categorical columns.<jupyter_code>ct_2 = ColumnTransformer([('categorical_features', OneHotEncoder(sparse=False, drop = 'first'), ['col_2', 'col_3'])])<jupyter_output><empty_output><jupyter_text>Similarly, you can all the names of the columns with categorical features.<jupyter_code>ct_2.fit_transform(data)
data<jupyter_output><empty_output><jupyter_text>Now let us take into consideration numerical features in this example, since preprocessing for numerical and categorical features will be different. For this example, we will use `MinMaxScaler` for numerical features, and I named it ‘numerical_features’.<jupyter_code>from sklearn.preprocessing import MinMaxScaler
ct_3 = ColumnTransformer([('categorical_features', OneHotEncoder(sparse=False, drop = 'first'), ['col_2', 'col_3']), 
                         ('numerical_features', MinMaxScaler(), ['col_1', 'col_4'])])
ct_3.fit_transform(data)
new_data = ct_3.fit_transform(data)<jupyter_output><empty_output><jupyter_text>I am inserting the transformed data into a data frame.<jupyter_code>pd.DataFrame(new_data)<jupyter_output><empty_output><jupyter_text>If you want to specific columns untouched, in this example the numerical features, you can use ‘passthrough’ keyword. You can also use 'drop' keyword to remove the columns.<jupyter_code>ct_4 = ColumnTransformer([('categorical_features', OneHotEncoder(sparse=False, drop = 'first'), ['col_2', 'col_3']), 
                         ('numerical_features', 'passthrough', ['col_1', 'col_4'])])
pd.DataFrame(ct_4.fit_transform(data))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1180089485458613, 'fraction_numerical': 0.010067114093959731, 'mean_word_length': 6.725701943844492, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '325316', 'license': 'MIT', 'n_tokens_mistral': 1147, 'n_tokens_neox': 1119, 'n_words': 349, 'path': 'Tutorial ColumnTransformer.ipynb', 'repo_name': 'chitralputhran/Tutorial-sklearn-ColumnTransformer', 'size': '3576'}