<jupyter_start><jupyter_text>Handle Categorical Features  importing librarries<jupyter_code>import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
import seaborn as sns
import stat

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Nominal EncodingNominal data is data that can be labelled or classified into mutually exclusive categories within a variable. These categories cannot be ordered in a meaningful way.For example, for the nominal variable of preferred mode of transportation, you may have the categories of car, bus, train, tram or bicycle.  1. One Hot Encoding___ Advantage:- easy to implement Disadvantage:- space. for every feature thre will be created n-1 unique features- curse of dimensionality___Always remember to drop 1 extra column otherwise it wil create multi-colinearity.  example 1<jupyter_code>df=pd.read_csv('titanic.csv',usecols=['Sex'])
df.head()
pd.get_dummies(df).head()
pd.get_dummies(df, drop_first=True).head() # to avoid multi-colinearity<jupyter_output><empty_output><jupyter_text>example 2<jupyter_code>df=pd.read_csv('titanic.csv',usecols=['Embarked'])
df['Embarked'].unique()
df.dropna(inplace=True) # just droping empty cells, otherwise we should fill na with "NA"
pd.get_dummies(df).head()
pd.get_dummies(df, drop_first=True).head()<jupyter_output><empty_output><jupyter_text>Advantages1. It is very simple to implement2. Does not increase the feature dimensional space Disadvantages1. If some of the labels have the same count, then they will be replaced with the same count and they will loose some valuable information.2 Adds somewhat arbitrary numbers, and therefore weights to the different labels, that may not be related to their predictive powerFollow this thread in Kaggle for more information:https://www.kaggle.com/general/16927  2. Onehot encoding with many categories in a feature___This technique used in KDD orange cup. Only need to allpy onehot encoding to top 10 most frequent labels or categories.___ KDD Cup Orange ChallengeWhat can we do instead?http://proceedings.mlr.press/v7/niculescu09/niculescu09.pdfIn the winning solution of the KDD 2009 cup: "Winning the KDD Cup Orange Challenge with Ensemble The Team suggested using 10 most frequent labels convert them into dummy variables using onehotencoding<jupyter_code>df=pd.read_csv('mercedes.csv',usecols=["X0","X1","X2","X3","X4","X5","X6"])
df.head()
count = 0
for col in df.columns:
    count += df[col].nunique()
    print(f"{col} have {df[col].nunique():>3} unique-cols")
count<jupyter_output>X0 have  47 unique-cols
X1 have  27 unique-cols
X2 have  44 unique-cols
X3 have   7 unique-cols
X4 have   4 unique-cols
X5 have  29 unique-cols
X6 have  12 unique-cols<jupyter_text>if we use normal one-hot encoding then it will create 170 number of extra columns which is a lot because we originally have 7 columns and from 7 to 170; 10X it will make model quality very bad as well as consuming a lot more space.<jupyter_code>df.X1.value_counts().sort_values(ascending=False).head(10)
lst_10 = df.X1.value_counts().sort_values(ascending=False).head(10).index
for index in df['X4'].value_counts().nlargest(10).index:
    print(index)
lst_10.shape
import numpy as np


def one_hot_encode_many(df, feature, drop_first=False):
    counts = df[feature].nunique()
    top_10 = list(df[feature].value_counts().nlargest(10).index)
    
    if counts <= 10:
        top_10.pop()
    
    for category in top_10:
        new_name = feature + f"_{category}"
        
        df[new_name] = np.where(df[feature]==category, 1, 0)
    
    if drop_first == True:
        dp.drop(columns=[feature], inplace=True)
one_hot_encode_many(df, "X1")
df.head()
one_hot_encode_many(df, "X0")
one_hot_encode_many(df, "X2")
one_hot_encode_many(df, "X3")
one_hot_encode_many(df, "X4")
one_hot_encode_many(df, "X5")
one_hot_encode_many(df, "X6")
df.head()
df.shape
df.drop(columns=['X0','X1','X2','X3','X4','X5','X6'], inplace=True)
df.head()
df.columns<jupyter_output><empty_output><jupyter_text>3. Mean Encoding___it it only applicable to classification problemsIn mean encoding we replace categorical values by count of 1s/ count of 1s and 0s per feature___ Pros of MeanEncoding:Capture information within the label, therefore rendering more predictive featuresCreates a monotonic relationship between the variable and the target Cons of MeanEncodig:It may cause over-fitting in the model.  example 1<jupyter_code>data={'SubjectName':['s1','s2','s3','s1','s4','s3','s2','s1','s2','s4','s1'],
      'Target':[1,0,1,1,1,0,0,1,1,1,0]}

df = pd.DataFrame(data)
df
df.groupby("SubjectName")['Target'].sum()
df.groupby("SubjectName")['Target'].mean()
x = df.groupby("SubjectName")['Target'].mean().to_dict()
x
df['SubjectName_mean'] = df['SubjectName'].map(x)
df
df.drop('SubjectName', axis=1)
def mean_encoding(df, feature, targetVar, drop_first=False):
    mean = df.groupby(feature)[targetVar].mean().to_dict()
    new_name = feature + '_mean'
    
    df[new_name] = df[feature].map(mean)
    
    if drop_first == True:
        df.drop(columns=[feature], inplace=True)
data={'SubjectName':['s1','s2','s3','s1','s4','s3','s2','s1','s2','s4','s1'],
      'Target':[1,0,1,1,1,0,0,1,1,1,0]}

df = pd.DataFrame(data)
df
mean_encoding(df, 'SubjectName', 'Target', drop_first=True)
df<jupyter_output><empty_output><jupyter_text>Count Encoding<jupyter_code>train_set = pd.read_csv('http://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data' , header = None,index_col=None)
train_set.head() 
train_set.shape
train_set.describe()
columns = [1,3,5,6,7,8,9,13]
train_set=train_set[columns]
train_set.head()
train_set.columns=['Employment','Degree','Status','Designation','family_job','Race','Sex','Country']
train_set.head()
for col in train_set.columns:
    print(col)
for col in train_set.columns:
    print(f"{col} : {train_set[col].nunique()}")
for col in train_set.columns:
    print(f"{col} : {train_set[col].unique()}")
    print("=" * 50)
train_set['Employment'] = train_set['Employment'].replace({' ?' : "MISSING"})
train_set['Designation'] = train_set['Designation'].replace({' ?' : "MISSING"})
train_set['Country'] = train_set['Country'].replace({' ?' : "MISSING"})
for col in train_set.columns:
    print(f"{col} : {train_set[col].unique()}")
    print("=" * 50)
train_set.isnull().sum()
train_set['Country'].value_counts().to_dict()
def count_encoding(df, feature):
    count_dict = train_set[feature].value_counts().to_dict()
    df[feature] = df[feature].map(count_dict)
count_encoding(train_set, 'Country')
train_set['Country'].unique()
train_set.head()
train_set['Country'].value_counts().to_dict()
for col in train_set.columns:
    count_encoding(train_set, col)
train_set.head()<jupyter_output><empty_output><jupyter_text>Advantages1. Easy To Use2. Not increasing feature space  Disadvantages1. It will provide same weight if the frequencies are same  Ordinal Encoding:____Ordinal data has two characteristics:The data can be classified into different categories within a variable.The categories have a natural ranked order.However, unlike with interval data, the distances between the categories are uneven or unknown.  1. label Encoding<jupyter_code>import datetime as dt
todays_date = dt.datetime.now()
todays_date
days = [todays_date - dt.timedelta(x) for x in range(15)]
days
df = pd.DataFrame(days, columns=['day'], dtype='datetime64[ns]')
df.head()
df['weekday'] = df['day'].dt.strftime("%A")
df.head()
df['weekday_ordinal'] = df['day'].dt.weekday
df.head()<jupyter_output><empty_output><jupyter_text>2. Target Guided Ordinal Encoding1. Ordering the labels according to the target2. Replace the labels by the joint probability of being 1 or 0<jupyter_code>df=pd.read_csv('titanic.csv', usecols=['Cabin','Survived'])
df.head()
df.isnull().sum()
df['Cabin'].fillna("MISSING", inplace=True)
df.head()
df.isnull().sum()
df['Cabin'].unique()
plt.figure(figsize=(20, 20))
sns.countplot(y = df['Cabin'])
plt.show()
df['Cabin'].value_counts()
df['Cabin'].nunique()
df['Cabin'].str[0]
df['Cabin_new'] = df['Cabin'].str[0]
df.head()
df['Cabin_new'].nunique()
df.groupby('Cabin_new')['Survived'].mean()
df.groupby('Cabin_new')['Survived'].mean().sort_values()
ordinal_labels = df.groupby('Cabin_new')['Survived'].mean().sort_values().index
ordinal_labels
{key:value for value, key in enumerate(ordinal_labels, 0)}
ordinal_encode = {key:value for value, key in enumerate(ordinal_labels, 0)}
df['Cabin_new'].map(ordinal_encode)
df['Cabin_new'] = df['Cabin_new'].map(ordinal_encode)
df.head()
def target_encoding(df, feature, targetVar):
    ordinal_labels = df.groupby(feature)[targetVar].sum().sort_values().index
    ordinal_encode = {key:value for value, key in enumerate(ordinal_labels, 0)}
    new_name = feature + '_tg_encode'
    df[new_name] = df[feature].map(ordinal_encode)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14216295626624478, 'fraction_numerical': 0.021471352695219798, 'mean_word_length': 6.218597063621534, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '563107', 'license': 'MIT', 'n_tokens_mistral': 3126, 'n_tokens_neox': 3013, 'n_words': 751, 'path': 'techniques with code/Different types of categorical encoding.ipynb', 'repo_name': 'ikaushikpal/Feature-Engineering-ML', 'size': '8849'}