<jupyter_start><jupyter_text>Handling Imbalanced Dataset with Machine Learning<jupyter_code>df = pd.read_csv("data/creditcard.csv")
df.head()
df.shape
df.isnull().sum()
# Check the dependent feature ---> y
df['Class'].value_counts()
# Independent and Dependent Features
# Split into X(features) and y(labels)

X=df.drop("Class",axis=1)
y=df['Class']<jupyter_output><empty_output><jupyter_text>Cross Validation Like KFOLD and Hyperparameter Tuning<jupyter_code># Lets apply a simple ML algorithm 

from sklearn.linear_model import LogisticRegression # Ml algorithm
from sklearn.metrics import accuracy_score,confusion_matrix,classification_report # evaluation methods

from sklearn.model_selection import KFold # cross-validation to help imbalanced datasets
from sklearn.model_selection import GridSearchCV # hyperparameter tuning
log_class = LogisticRegression()

# Hyperparameter tuning
grid = {
    'C':10.0 **np.arange(-2,3), # giving C different values
    'penalty':['l1','l2']
}
cv=KFold(n_splits=5,random_state=None,shuffle=False)
from sklearn.model_selection import train_test_split

np.random.seed(42)

X_train,X_test,y_train,y_test=train_test_split(X,y,train_size=0.7)
clf=GridSearchCV(log_class,grid,cv=cv,n_jobs=-1,scoring='f1_macro')
clf.fit(X_train,y_train)
# Make predictions with the best hyperparameters
y_pred=clf.predict(X_test)

# Evaluate the predictions
print(confusion_matrix(y_test,y_pred))
print(accuracy_score(y_test,y_pred))
print(classification_report(y_test,y_pred))<jupyter_output>[[85253    54]
 [   48    88]]
0.9988062216916541
              precision    recall  f1-score   support

           0       1.00      1.00      1.00     85307
           1       0.62      0.65      0.63       136

    accuracy                           1.00     85443
   macro avg       0.81      0.82      0.82     85443
weighted avg       1.00      1.00      1.00     85443<jupyter_text>The number of 1s are very less compared to the number of 0s, the ratio difference is huge....so the dataset is imbalanced<jupyter_code>X_train.shape
y_train.value_counts()<jupyter_output><empty_output><jupyter_text>1. Under Sampling0  ----------------      110000  -------  100**undersampling** -> reduce the points of maximum labelsThere are a lot of disadvantages in using this method, due to loss of data.Used: only when u have **small datasets**<jupyter_code>y_train.value_counts()
from collections import Counter
Counter(y_train)
# we will reduce 0s by 80%

from imblearn.under_sampling import RandomUnderSampler  
under_sampler = RandomUnderSampler(0.8) 
X_train_ns, y_train_ns = under_sampler.fit_resample(X_train, y_train) # get new data set

print("The number of classes before fit {}".format(Counter(y_train)))
print("The number of classes after fit {}".format(Counter(y_train_ns)))
# Train model with new dataset(after undersampling)

from sklearn.ensemble import RandomForestClassifier
classifier=RandomForestClassifier()
classifier.fit(X_train_ns,y_train_ns)
y_pred=classifier.predict(X_test)
print(confusion_matrix(y_test,y_pred))
print(accuracy_score(y_test,y_pred))
print(classification_report(y_test,y_pred))

# As you can see, precision(accuracy) of 1 has gone to hell! Also, there are a lot of false negatives<jupyter_output>[[84044  1263]
 [   10   126]]
0.9851011785634868
              precision    recall  f1-score   support

           0       1.00      0.99      0.99     85307
           1       0.09      0.93      0.17       136

    accuracy                           0.99     85443
   macro avg       0.55      0.96      0.58     85443
weighted avg       1.00      0.99      0.99     85443<jupyter_text>2. Over Sampling* whichever classess has less number of values, we will impute and increase the values in those classes<jupyter_code>from imblearn.over_sampling import RandomOverSampler

os=RandomOverSampler(0.75) # 75%
X_train_ns,y_train_ns = os.fit_resample(X_train,y_train)

print("The number of classes before fit {}".format(Counter(y_train)))
print("The number of classes after fit {}".format(Counter(y_train_ns)))
# Train model with new dataset(after oversampling)

from sklearn.ensemble import RandomForestClassifier
classifier=RandomForestClassifier()
classifier.fit(X_train_ns,y_train_ns)
y_pred=classifier.predict(X_test)
print(confusion_matrix(y_test,y_pred))
print(accuracy_score(y_test,y_pred))
print(classification_report(y_test,y_pred))

# By oversampling, false negative has been reduced. Also, precision increases.<jupyter_output>[[85301     6]
 [   24   112]]
0.9996488887328394
              precision    recall  f1-score   support

           0       1.00      1.00      1.00     85307
           1       0.95      0.82      0.88       136

    accuracy                           1.00     85443
   macro avg       0.97      0.91      0.94     85443
weighted avg       1.00      1.00      1.00     85443<jupyter_text>3. SMOTETomek* it creates more number of points(new points) of the lowest class* This method takes more time<jupyter_code>from imblearn.over_sampling import SMOTE

oversample = SMOTE(0.75)
X_train_ns,y_train_ns = oversample.fit_resample(X_train,y_train)

print("The number of classes before fit {}".format(Counter(y_train)))
print("The number of classes after fit {}".format(Counter(y_train_ns)))
# Train model with new dataset

from sklearn.ensemble import RandomForestClassifier
classifier=RandomForestClassifier()
classifier.fit(X_train_ns,y_train_ns)
y_pred=classifier.predict(X_test)
print(confusion_matrix(y_test,y_pred))
print(accuracy_score(y_test,y_pred))
print(classification_report(y_test,y_pred))<jupyter_output>[[85281    26]
 [   17   119]]
0.9994967405170698
              precision    recall  f1-score   support

           0       1.00      1.00      1.00     85307
           1       0.82      0.88      0.85       136

    accuracy                           1.00     85443
   macro avg       0.91      0.94      0.92     85443
weighted avg       1.00      1.00      1.00     85443<jupyter_text>4. Ensemble Techniques* it creates more number of points(new points) of the lowest class<jupyter_code>from imblearn.ensemble import EasyEnsembleClassifier

easy=EasyEnsembleClassifier()
easy.fit(X_train,y_train)
y_pred=easy.predict(X_test)
print(confusion_matrix(y_test,y_pred))
print(accuracy_score(y_test,y_pred))
print(classification_report(y_test,y_pred))<jupyter_output>[[82087  3220]
 [    8   128]]
0.9622204276535234
              precision    recall  f1-score   support

           0       1.00      0.96      0.98     85307
           1       0.04      0.94      0.07       136

    accuracy                           0.96     85443
   macro avg       0.52      0.95      0.53     85443
weighted avg       1.00      0.96      0.98     85443
{'fraction_non_alphanumeric': 0.08882947118241236, 'fraction_numerical': 0.0766488413547237, 'mean_word_length': 3.130674846625767, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '540185', 'license': 'MIT', 'n_tokens_mistral': 2525, 'n_tokens_neox': 2247, 'n_words': 532, 'path': 'Imbalanced Data/Handling_Imbalanced_Data.ipynb', 'repo_name': 'Jawwad-Fida/Feature-Engineering', 'size': '6732'}