<jupyter_start><jupyter_text>Handson  of Feature Scaling - Part2Youtube Explanation : https://youtu.be/KqX1X6ckf-4 - sklearn - Min-Max Scaler - Normalization - sklearn - Standard Scaler - Standardization - sklearn - RobustScaler  Which technique we should use Normalization or Standardization?  - Normalization is good to use when you know that the distribution of your data does not follow a Gaussian distribution. This can be useful in algorithms that do not assume any distribution of the data like K-Nearest Neighbors and Neural Networks.  - Standardization, on the other hand, can be helpful in cases where the data follows a Gaussian distribution. However, this does not have to be necessarily true. Also, unlike normalization, standardization does not have a bounding range. So, even if you have outliers in your data, they will not be affected by standardization.  - There is no any thumb rule to tell you when to normalize or standardize your data. You can always start by fitting your model to raw, normalized and standardized data and compare the performance for best results.  **It is a good practice to fit the scaler on the training data and then use it to transform the testing data. This would avoid any data leakage during the model testing process.*<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline 
# load the other dataset
df_country = pd.read_excel("https://raw.githubusercontent.com/atulpatelDS/Data_Files/master/Feature_Engineering/Scaling.xlsx")
df_country                      
X = df_country.drop("Country_Type",axis=1)
y = df_country["Country_Type"]
print(X.shape)
print(y.shape)
print(type(X))
print(type(y))
from sklearn.model_selection import train_test_split
# First we will devide the dataset in train and test
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.33, random_state=42)
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)
df_exp = X_train
df_exp
X_test<jupyter_output><empty_output><jupyter_text>Min-Max Scaler<jupyter_code>from sklearn.preprocessing import MinMaxScaler
# Create the object
minmax_scale = MinMaxScaler()
# fit on the train dataset
minmax_scale.fit((df_exp))
# transform the train dataset columns ("Experience" and "Salary")
df_exp_minmax = minmax_scale.transform(df_exp)
print(type(df_exp_minmax))
df_exp_minmax_new = pd.DataFrame(df_exp_minmax,columns=["Experience","Salary"])
df_exp_minmax_new
# To avoid the data leakage we fisrt split the dataset then wecreate object and fit of train dataset
# and use that fit to transform the test dataset
# transform the test dataset using train dataset fit ("Experience" and "Salary")
X_test_minmax = minmax_scale.transform(X_test)
print(type(X_test_minmax))
df_test_minmax_new = pd.DataFrame(X_test_minmax,columns=["Experience","Salary"])
df_test_minmax_new
ax = df_exp.plot.scatter(x="Experience", y="Salary",marker = "P",s=80, label='BREFORE SCALING',color='red')
df_exp_minmax_new.plot.scatter(x="Experience", y="Salary",marker = "x",s=60,label='AFTER SCALING',
                               color='blue', ax = ax)
plt.axhline(0, color='red',alpha=0.2)
plt.axvline(0, color='red',alpha=0.2)<jupyter_output><empty_output><jupyter_text>Standard Scaler<jupyter_code>from sklearn.preprocessing import StandardScaler
# create the object
std_scale = StandardScaler()
# fit and transform the columns ("Experience" and "Salary")
df_exp_std = std_scale.fit_transform(df_exp.iloc[:,0:2])
print(type(df_exp_std))
df_exp_std_new = pd.DataFrame(df_exp_std,columns=["Experience","Salary"])
df_exp_std_new
print("std: ",round(df_exp_std_new.Experience.std(),0))
print("mean: ",round(df_exp_std_new.Experience.mean(),2))<jupyter_output>std:  1.0
mean:  -0.0
{'fraction_non_alphanumeric': 0.09382454280413464, 'fraction_numerical': 0.0074211502782931356, 'mean_word_length': 5.912087912087912, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '155644', 'license': 'MIT', 'n_tokens_mistral': 1220, 'n_tokens_neox': 1199, 'n_words': 368, 'path': 'Feature_Engineering/Feature Scaling - Sklearn Min-Max Scaler and Standard ScalerTutorial 2.ipynb', 'repo_name': 'atulpatelDS/Youtube', 'size': '3773'}