<jupyter_start><jupyter_text>Missing Values : Imputation with Sklearn Preprocessing  ------------------------------------------------------------------------------------------------------------------------------- https://www.youtube.com/watch?v=3Pj6OoGrdVM    ---->  Handling Missing Values Using Imputerhttps://www.youtube.com/watch?v=ENvSybznF_o&feature=youtu.be  ---->  fit, transform and fit_transformhttps://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.iloc.htmlhttps://pandas.pydata.org/pandas-docs/version/0.17/generated/pandas.DataFrame.mode.html<jupyter_code>#  By making use of fit, the imputer calculates the mean of columns from the given data

#  transform() method then it applies those means to that data i.e. replacing missing values with the means.<jupyter_output><empty_output><jupyter_text>1. Numerical Missing Values              ------->    mean, median 2. Categorical Missing Values            ------->    mode<jupyter_code>Imputation with Sklearn:


       from sklearn.preprocessing import Imputer

       imputer = Imputer(missing_values = 'NaN', strategy='mean', axis=0)

       imputer.fit(x[:,[1,6]])

       x[:,[1,6]] = imputer.fit_transform(x[:,[1,6]])<jupyter_output><empty_output><jupyter_text>-------------------------------------------------------------------------------------------------------------------------------<jupyter_code>import pandas as pd
import numpy as np
df = pd.read_csv("F:/03. Suresh/1. Material/05. Data Science/14. Jupyter/08. Data Sets/Datapreprocessing.csv")
df
df.shape
df.isnull().sum()
x = df.iloc[:,:-1].values
y = df.iloc[:,-1].values<jupyter_output><empty_output><jupyter_text>Numerical Missing Values (mean, median)<jupyter_code>from sklearn.preprocessing import Imputer
imputer = Imputer(missing_values = 'NaN', strategy='mean', axis=0)
imputer.fit(x[:,[1,6]])
x[:,[1,6]] = imputer.fit_transform(x[:,[1,6]])
df1 = pd.DataFrame(x)
df1<jupyter_output><empty_output><jupyter_text>Categorical Missing Values (mode)<jupyter_code>cols = ['Occupation', 'Employment Status', 'Employement Type']
df[cols] = df[cols].fillna(df.mode().iloc[0])
df<jupyter_output><empty_output><jupyter_text>-------------------------------------------------------------------------------------------------------------------------------<jupyter_code>df['Occupation'].mode()
df['Occupation'].mode().iloc[0]
df['Occupation'].mode()[0]
df.mode()
# will return the mode of each column

df.mode().iloc[0]
# will fill the missing values with the mode of each columns

df[cols].fillna(df.mode().iloc[0])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.28074245939675174, 'fraction_numerical': 0.015081206496519721, 'mean_word_length': 2.5341530054644807, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 29, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '373483', 'license': 'CNRI-Python', 'n_tokens_mistral': 834, 'n_tokens_neox': 805, 'n_words': 148, 'path': '10_Missing_Values/2_Imputation_Sklearn/5_Imputation_Preprocessing_Sklearn/1_imputation_preprocessing.ipynb', 'repo_name': 'sureshmecad/Pandas', 'size': '2586'}