<jupyter_start><jupyter_text>OneHotEncoderPerforms One Hot Encoding.The encoder can select how many different labels per variable to encode into binaries. When top_categories is set to None, all the categories will be transformed in binary variables. However, when top_categories is set to an integer, for example 10, then only the 10 most popular categories will be transformed into binary, and the rest will be discarded.The encoder has also the possibility to create binary variables from all categories (drop_last = False), or remove the binary for the last category (drop_last = True), for use in linear models.<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from feature_engine.encoding import OneHotEncoder
# Load titanic dataset from OpenML

def load_titanic():
    data = pd.read_csv('https://www.openml.org/data/get_csv/16826755/phpMYEkMl')
    data = data.replace('?', np.nan)
    data['cabin'] = data['cabin'].astype(str).str[0]
    data['pclass'] = data['pclass'].astype('O')
    data['age'] = data['age'].astype('float')
    data['fare'] = data['fare'].astype('float')
    data['embarked'].fillna('C', inplace=True)
    data.drop(labels=['boat', 'body', 'home.dest'], axis=1, inplace=True)
    return data
data = load_titanic()
data.head()
X = data.drop(['survived', 'name', 'ticket'], axis=1)
y = data.survived
# we will encode the below variables, they have no missing values
X[['cabin', 'pclass', 'embarked']].isnull().sum()
''' Make sure that the variables are type (object).
if not, cast it as object , otherwise the transformer will either send an error (if we pass it as argument) 
or not pick it up (if we leave variables=None). '''

X[['cabin', 'pclass', 'embarked']].dtypes
# let's separate into training and testing set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

X_train.shape, X_test.shape<jupyter_output><empty_output><jupyter_text>One hot encoding consists in replacing the categorical variable by acombination of binary variables which take value 0 or 1, to indicate ifa certain category is present in an observation.Each one of the binary variables are also known as dummy variables. Forexample, from the categorical variable "Gender" with categories 'female'and 'male', we can generate the boolean variable "female", which takes 1if the person is female or 0 otherwise. We can also generate the variablemale, which takes 1 if the person is "male" and 0 otherwise.The encoder has the option to generate one dummy variable per category, orto create dummy variables only for the top n most popular categories, that is,the categories that are shown by the majority of the observations.If dummy variables are created for all the categories of a variable, you havethe option to drop one category not to create information redundancy. That is,encoding into k-1 variables, where k is the number if unique categories.The encoder will encode only categorical variables (type 'object'). A listof variables can be passed as an argument. If no variables are passed as argument, the encoder will find and encode categorical variables (object type). Note:New categories in the data to transform, that is, those that did not appearin the training set, will be ignored (no binary variable will be created for them).  All binary, no top_categories<jupyter_code>'''
Parameters
----------

top_categories: int, default=None
    If None, a dummy variable will be created for each category of the variable.
    Alternatively, top_categories indicates the number of most frequent categories
    to encode. Dummy variables will be created only for those popular categories
    and the rest will be ignored. Note that this is equivalent to grouping all the
    remaining categories in one group.
    
variables : list
    The list of categorical variables that will be encoded. If None, the  
    encoder will find and select all object type variables.
    
drop_last: boolean, default=False
    Only used if top_categories = None. It indicates whether to create dummy
    variables for all the categories (k dummies), or if set to True, it will
    ignore the last variable of the list (k-1 dummies).
'''
ohe_enc = OneHotEncoder(top_categories=None,
                        variables=['pclass', 'cabin', 'embarked'],
                        drop_last=False)
ohe_enc.fit(X_train)
ohe_enc.encoder_dict_
train_t = ohe_enc.transform(X_train)
test_t = ohe_enc.transform(X_train)

test_t.head()<jupyter_output><empty_output><jupyter_text>Selecting top_categories to encode<jupyter_code>ohe_enc = OneHotEncoder(top_categories=2,
                        variables=['pclass', 'cabin', 'embarked'],
                        drop_last=False)
ohe_enc.fit(X_train)
ohe_enc.encoder_dict_
train_t = ohe_enc.transform(X_train)
test_t = ohe_enc.transform(X_train)
test_t.head()<jupyter_output><empty_output><jupyter_text>Dropping the last category for linear models<jupyter_code>ohe_enc = OneHotEncoder(top_categories=None,
                        variables=['pclass', 'cabin', 'embarked'],
                        drop_last=True)

ohe_enc.fit(X_train)
ohe_enc.encoder_dict_
train_t = ohe_enc.transform(X_train)
test_t = ohe_enc.transform(X_train)

test_t.head()<jupyter_output><empty_output><jupyter_text>Automatically select categorical variablesThis encoder selects all the categorical variables, if None is passed to the variable argument when calling the encoder.<jupyter_code>ohe_enc = OneHotEncoder(top_categories=None,
                        drop_last=True)

ohe_enc.fit(X_train)
train_t = ohe_enc.transform(X_train)
test_t = ohe_enc.transform(X_train)

test_t.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08355136926565498, 'fraction_numerical': 0.004709576138147566, 'mean_word_length': 4.594146341463414, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '10185', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1682, 'n_tokens_neox': 1620, 'n_words': 634, 'path': 'examples/encoding/OneHotEncoder.ipynb', 'repo_name': 'indymnv/feature_engine', 'size': '5733'}