<jupyter_start><jupyter_text>Fraud Prediction Using Skewed Data How do we achieve good predictive power in the statistical model? Can sampling techniques help? Let's find out!  1. SetupTo prepare your environment, you need to install some packages. 1.1 Install the necessary packagesYou need the latest versions of these packages:<jupyter_code>import numpy as np 
import pandas as pd 
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>2. Read the Data & convert it into DataframeClick on Insert to code and then select Insert pandas DataFrame in the below empty cell.  3. Assign a new name for the dataframe<jupyter_code>'''Rename the dataframe to df'''

df = df_data_2<jupyter_output><empty_output><jupyter_text>4. Descriptive statistics on the data<jupyter_code>print("Time")
print(df.Time.describe())
print("V1")
print(df.Time.describe())
print("Amount")
print(df.Amount.describe())
print("Class")
print(df.Class.describe())
print(df.Class.nunique())
print("ALL")
print(df.describe())<jupyter_output>Time
count     56874.000000
mean      94995.469934
std       47535.817923
min          10.000000
25%       54380.250000
50%       84664.500000
75%      139664.500000
max      172782.000000
Name: Time, dtype: float64
V1
count     56874.000000
mean      94995.469934
std       47535.817923
min          10.000000
25%       54380.250000
50%       84664.500000
75%      139664.500000
max      172782.000000
Name: Time, dtype: float64
Amount
count    56874.000000
mean        88.264681
std        233.922284
min          0.000000
25%          5.740000
50%         22.205000
75%         78.600000
max       7862.390000
Name: Amount, dtype: float64
Class
count    56874.000000
mean         0.001793
std          0.042311
min          0.000000
25%          0.000000
50%          0.000000
75%          0.000000
max          1.000000
Name: Class, dtype: float64
2
ALL
                Time            V1            V2            V3            V4  \
count   56874.000000  56874.000000  56874.000000  56874.000000[...]<jupyter_text>5. Use Histogram to visualize<jupyter_code>df.Time.hist()
df.Class.hist()
df['Amount'].hist(by=df['Class'])
print(df.groupby('Class').size())
df.hist(figsize=(15,15))<jupyter_output><empty_output><jupyter_text>6. Check the spread of fraud vs non-fraud on selected variables This is done to identify variables selection which has good spread of frauds and non frauds<jupyter_code>sample = df[df['Class']==0]
fraud = df[df['Class']==1]
print('V9 - V10')
plt.scatter(fraud['V9'], fraud['V10'],s=1, color='r')
plt.scatter(sample['V9'], sample['V10'], s=1, color='g')
plt.show()
plt.clf()
print('V16-V17')
plt.scatter(sample['V16'], sample['V17'], s=1, color = 'g')
plt.scatter(fraud['V16'], fraud['V17'], s=1, color = 'r')
plt.show()
plt.clf()
print('V17 - V18')
plt.scatter(sample['V18'], sample['V17'], s=1, color = 'g')
plt.scatter(fraud['V18'], fraud['V17'], s=1, color = 'r')
plt.show()
plt.clf()
print('V1 - V3')
plt.scatter(sample['V1'], sample['V3'], s=1, color = 'g')
plt.scatter(fraud['V1'], fraud['V3'], s=1, color = 'r')
plt.show()
plt.clf()
print('V1 - V2')
plt.scatter(sample['V1'], sample['V2'], s=1, color = 'g')
plt.scatter(fraud['V1'], fraud['V2'], s=1, color = 'r')
plt.show()
plt.clf()<jupyter_output>V1 - V2<jupyter_text>7. Split the data into train & test data sets using 70:30 mix The model will be built on training data and will be applied on the test data<jupyter_code># Split The Data with all variables

from sklearn.model_selection import train_test_split

x = df[['Time', 'V1', 'V2', 'V3', 'V4', 'V5', 'V6', 'V7', 'V8', 'V9', 'V10',
       'V11', 'V12', 'V13', 'V14', 'V15', 'V16', 'V17', 'V18', 'V19', 'V20',
       'V21', 'V22', 'V23', 'V24', 'V25', 'V26', 'V27', 'V28', 'Amount']]
y = df['Class']

xtrain, xtest, ytrain, ytest = train_test_split(x, y, test_size=0.30, random_state=0)
print('xtrain shape')
print(xtrain.shape)
print('xtest shape')
print(xtest.shape)<jupyter_output>xtrain shape
(39811, 30)
xtest shape
(17063, 30)<jupyter_text>8. Use Random Forest Algorithm A brief about Random Forest AlgorithmRandom forest classifier creates a set of decision trees from randomly selected subset of training set. It then aggregates the votes from different decision trees to decide the final class of the test object.  Random forests or random decision forests are an ensemble learning method for classification, regression and other tasks, that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. Random decision forests will avoid overfitting which will enhance the accuracy of the model on new data. This is a Bagging based algorithm which is used for reducing Overfitting in order to create strong learners for generating accurate predictions.<jupyter_code># RF Classifier explained

'''rf = RandomForestClassifier(n_estimators=100, oob_score=True, n_jobs=4)
n_estimators : integer, optional (default=10)
The number of trees in the forest.
oob_score : bool (default=False)
Whether to use out-of-bag samples to estimate the generalization accuracy.
n_jobs : integer, optional (default=1)
The number of jobs to run in parallel for both fit and predict. If -1, then the number of jobs is set to the number of cores.
'''

from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics

rfmodel = RandomForestClassifier()
rfmodel.fit(xtrain,ytrain)
print('model')
print(rfmodel)

ypredrf = rfmodel.predict(xtest)
print('confusion matrix')
print(metrics.confusion_matrix(ytest, ypredrf))
print('classification report')
print(metrics.classification_report(ytest, ypredrf))
print('Accuracy : %f' % (metrics.accuracy_score(ytest, ypredrf)))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, ypredrf)))<jupyter_output>model
RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
            max_depth=None, max_features='auto', max_leaf_nodes=None,
            min_impurity_decrease=0.0, min_impurity_split=None,
            min_samples_leaf=1, min_samples_split=2,
            min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,
            oob_score=False, random_state=None, verbose=0,
            warm_start=False)
confusion matrix
[[17029     1]
 [    9    24]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17030
          1       0.96      0.73      0.83        33

avg / total       1.00      1.00      1.00     17063

Accuracy : 0.999414
Area under the curve : 0.863607<jupyter_text>In the above classification report we are getting the f1-score of 89 which is the harmonic mean of precision & recall scores. Recall is where the model tries to recollect the number of instances, in this case the model has been able to recollect 82% of frauds and able to classify them as frauds 96 % of the recollected events which is the precision of the model.  Area under the curve signifies the accuracy of the model (values between 0 & 1) where the score towards 1 indicate high predictive power.  Note  : The numbers for recall, precision, F1 Score & Area under the curve can change for different runs of the model due to stocastic nature of the algorithms.  9. Use Gradient Boosting Alogrithm A Brief about Gradient Boosting AlgorithmGradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees. It builds the model in a stage-wise fashion like other boosting methods do, and it generalizes them by allowing optimization of an arbitrary differentiable loss function(a loss function or cost function is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost" associated with the event). This is a Boosting based algorithm which is an ensemble technique to combine weak learners to create a strong learner that can make accurate predictions.<jupyter_code># GBM Classifier explained

'''params = {'n_estimators': 500, 'max_depth': 3, 'subsample': 0.5, 'learning_rate': 0.01, 'min_samples_leaf': 1, 'random_state': 3}
clf = ensemble. GradientBoostingClassifier(**params)
n_estimators : int (default=100)
The number of boosting stages to perform. Gradient boosting is fairly robust to over-fitting, so a large number usually results in better performance.
max_depth: integer, optional (default=3)
maximum depth of the individual regression estimators. The maximum depth limits the number of nodes in the tree. The best value depends on the interaction of the input variables.
subsample: float, optional (default=1.0)
The fraction of samples to be used for fitting the individual base learners. If smaller than 1.0 this results in Stochastic Gradient Boosting. subsample interacts with the parameter n_estimators. Choosing subsample < 1.0 leads to a reduction of variance and an increase in bias.
random_state : int, RandomState instance or None, optional (default=None)
If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random.
learning_rate : float, optional (default=0.1)
learning rate shrinks the contribution of each tree by learning_rate. There is a trade-off between learning_rate and n_estimators.
min_samples_leaf : int, float, optional (default=1)
The minimum number of samples required to be at a leaf node:
'''

from sklearn import ensemble

params = {'n_estimators': 500, 'max_depth': 3, 'subsample': 0.5,
          'learning_rate': 0.01, 'min_samples_leaf': 1, 'random_state': 3}
clf = ensemble.GradientBoostingClassifier(**params)
clf.fit(xtrain, ytrain) #trains
y_pred = clf.predict(xtest)  #predicts
print('confusion matrix')
print(metrics.confusion_matrix(ytest, y_pred))
print('classification report')
print(metrics.classification_report(ytest, y_pred))
print("-----------------------------------------------------------------------------------------")
print("Accuracy is :")
print(metrics.accuracy_score(ytest, y_pred))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, y_pred)))<jupyter_output>confusion matrix
[[17022     8]
 [   10    23]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17030
          1       0.74      0.70      0.72        33

avg / total       1.00      1.00      1.00     17063

-----------------------------------------------------------------------------------------
Accuracy is :
0.998945085858
Area under the curve : 0.848250<jupyter_text>10. Use Extreme Gradient Boosting Alogrithm A Brief about Extreme Gradient Boosting AlgorithmXGBoost is one of the implementations of Gradient Boosting concept, but what makes XGBoost unique is that it uses “a more regularized model formalization to control over-fitting, which gives it better performance,” according to the author of the algorithm, <NAME>. Therefore, it helps to reduce overfitting.<jupyter_code>from xgboost.sklearn import XGBClassifier

# Create the XGB classifier, xgb_model.
xgb_model = XGBClassifier()
# List the default parameters.
print(xgb_model.get_xgb_params())
# Train and evaluate.
xgb_model.fit(xtrain, ytrain, eval_metric=['error'], eval_set=[((xtrain, ytrain)),(xtest, ytest)])
from sklearn import metrics
import warnings
warnings.filterwarnings("ignore")

y_pred = xgb_model.predict(xtest)  #predicts
print('confusion matrix')
print(metrics.confusion_matrix(ytest, y_pred))
print('classification report')
print(metrics.classification_report(ytest, y_pred))
print("-----------------------------------------------------------------------------------------")
print("Accuracy is :")
print(metrics.accuracy_score(ytest, y_pred))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, y_pred)))<jupyter_output>confusion matrix
[[17028     2]
 [    7    26]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17030
          1       0.93      0.79      0.85        33

avg / total       1.00      1.00      1.00     17063

-----------------------------------------------------------------------------------------
Accuracy is :
0.999472542929
Area under the curve : 0.893881<jupyter_text>As we can see, the F1 score and Area under the curve are higher compared to previous model.   11. Select random variables for model building This is done to check whether we can get a lift on the accuracy with fewer variables<jupyter_code># Split The Data with few variables

from sklearn.model_selection import train_test_split

x = df[['V9', 'V10','V16', 'V17', 'V18', 'Amount']]
y = df['Class']

xtrain, xtest, ytrain, ytest = train_test_split(x, y, test_size=0.30, random_state=0)
print('xtrain shape')
print(xtrain.shape)
print('xtest shape')
print(xtest.shape)
'''Random Forest Classifier on reduced dimensions data'''

from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics

rfmodel = RandomForestClassifier()
rfmodel.fit(xtrain,ytrain)
print('model')
print(rfmodel)

ypredrf = rfmodel.predict(xtest)
print('confusion matrix')
print(metrics.confusion_matrix(ytest, ypredrf))
print('classification report')
print(metrics.classification_report(ytest, ypredrf))
print('Accuracy : %f' % (metrics.accuracy_score(ytest, ypredrf)))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, ypredrf)))<jupyter_output>model
RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
            max_depth=None, max_features='auto', max_leaf_nodes=None,
            min_impurity_decrease=0.0, min_impurity_split=None,
            min_samples_leaf=1, min_samples_split=2,
            min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,
            oob_score=False, random_state=None, verbose=0,
            warm_start=False)
confusion matrix
[[17028     2]
 [    8    25]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17030
          1       0.93      0.76      0.83        33

avg / total       1.00      1.00      1.00     17063

Accuracy : 0.999414
Area under the curve : 0.878729<jupyter_text>The F1 score has reduced with fewer variables along with Area under the curve<jupyter_code>'''Gradient Boost Algorithm on reduced dimensions data'''

from sklearn import ensemble

params = {'n_estimators': 500, 'max_depth': 3, 'subsample': 0.5,
          'learning_rate': 0.01, 'min_samples_leaf': 1, 'random_state': 3}
clf = ensemble.GradientBoostingClassifier(**params)
clf.fit(xtrain, ytrain) #trains
y_pred = clf.predict(xtest)  #predicts
print('confusion matrix')
print(metrics.confusion_matrix(ytest, y_pred))
print('classification report')
print(metrics.classification_report(ytest, y_pred))
print("-----------------------------------------------------------------------------------------")
print("Accuracy is :")
print(metrics.accuracy_score(ytest, y_pred))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, y_pred)))<jupyter_output>confusion matrix
[[17023     7]
 [    8    25]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17030
          1       0.78      0.76      0.77        33

avg / total       1.00      1.00      1.00     17063

-----------------------------------------------------------------------------------------
Accuracy is :
0.999120904882
Area under the curve : 0.878582<jupyter_text>Reduction in the F1 score with fewer variables.<jupyter_code>'''Extreme Gradient Boost Algorithm on reduced dimensions data'''

from xgboost.sklearn import XGBClassifier

# Create the XGB classifier, xgb_model.
xgb_model = XGBClassifier()
# List the default parameters.
print(xgb_model.get_xgb_params())
# Train and evaluate.
xgb_model.fit(xtrain, ytrain, eval_metric=['error'], eval_set=[((xtrain, ytrain)),(xtest, ytest)])
from sklearn import metrics
import warnings
warnings.filterwarnings("ignore")

y_pred = xgb_model.predict(xtest)  #predicts
print('confusion matrix')
print(metrics.confusion_matrix(ytest, y_pred))
print('classification report')
print(metrics.classification_report(ytest, y_pred))
print("-----------------------------------------------------------------------------------------")
print("Accuracy is :")
print(metrics.accuracy_score(ytest, y_pred))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, y_pred)))<jupyter_output>confusion matrix
[[17027     3]
 [    7    26]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17030
          1       0.90      0.79      0.84        33

avg / total       1.00      1.00      1.00     17063

-----------------------------------------------------------------------------------------
Accuracy is :
0.999413936588
Area under the curve : 0.893851<jupyter_text>Higher F1 score compared to Gradient Boosting Model  11. Random Under Sampling This is done to reduce the imbalance between frauds & non frauds<jupyter_code>'''We are taking 10% of non frauds and merge it with frauds data'''

df = df[['Class', 'Time', 'V1', 'V2', 'V3', 'V4', 'V5', 'V6', 'V7', 'V8', 'V9', 'V10',
       'V11', 'V12', 'V13', 'V14', 'V15', 'V16', 'V17', 'V18', 'V19', 'V20',
       'V21', 'V22', 'V23', 'V24', 'V25', 'V26', 'V27', 'V28', 'Amount']]


sample = df[df['Class']==0]
fraud = df[df['Class'] == 1]

# random sampling
ignore_me, sample = train_test_split(sample, test_size = 0.1)
import warnings
warnings.filterwarnings("ignore")

sample = pd.concat([sample, fraud])

# Split into train and test units.
xtrain, xtest = train_test_split(sample, test_size = 0.3)
ytrain = xtrain['Class']
ytest = xtest['Class']
xtrain.drop('Class', 1, inplace = True)
xtest.drop('Class', 1, inplace = True)
'''Random Forest Classifier on undersampled data'''

from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics

rfmodel = RandomForestClassifier()
rfmodel.fit(xtrain,ytrain)
print('model')
print(rfmodel)

ypredrf = rfmodel.predict(xtest)
print('confusion matrix')
print(metrics.confusion_matrix(ytest, ypredrf))
print('classification report')
print(metrics.classification_report(ytest, ypredrf))
print('Accuracy : %f' % (metrics.accuracy_score(ytest, ypredrf)))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, ypredrf)))<jupyter_output>model
RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
            max_depth=None, max_features='auto', max_leaf_nodes=None,
            min_impurity_decrease=0.0, min_impurity_split=None,
            min_samples_leaf=1, min_samples_split=2,
            min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,
            oob_score=False, random_state=None, verbose=0,
            warm_start=False)
confusion matrix
[[1703    0]
 [   2   29]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00      1703
          1       1.00      0.94      0.97        31

avg / total       1.00      1.00      1.00      1734

Accuracy : 0.998847
Area under the curve : 0.967742<jupyter_text>we can see an increase in F1 score which indicates that random under sample can give a lift in the accuracy. However we need to do random sampling multiple times for training the model and then use it on the new data for better results.<jupyter_code>'''Gradient Boost Algorithm on undersampled data'''

from sklearn import ensemble

params = {'n_estimators': 650, 'max_depth': 3, 'subsample': 0.5,
          'learning_rate': 0.01, 'min_samples_leaf': 1, 'random_state': 3}
clf = ensemble.GradientBoostingClassifier(**params)
clf.fit(xtrain, ytrain) #trains
y_pred = clf.predict(xtest)  #predicts
print('confusion matrix')
print(metrics.confusion_matrix(ytest, y_pred))
print('classification report')
print(metrics.classification_report(ytest, y_pred))
print("-----------------------------------------------------------------------------------------")
print("Accuracy is :")
print(metrics.accuracy_score(ytest, y_pred))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, y_pred)))<jupyter_output>confusion matrix
[[1700    3]
 [   3   28]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00      1703
          1       0.90      0.90      0.90        31

avg / total       1.00      1.00      1.00      1734

-----------------------------------------------------------------------------------------
Accuracy is :
0.996539792388
Area under the curve : 0.950732<jupyter_text>There is an increase in the F1 score for this sampling technique with 10% data of non-frauds and 100% frauds.<jupyter_code>'''Extreme Gradient Boost Algorithm on reduced dimensions data'''

from xgboost.sklearn import XGBClassifier

# Create the XGB classifier, xgb_model.
xgb_model = XGBClassifier()

# List the default parameters.
print(xgb_model.get_xgb_params())
# Train and evaluate.
xgb_model.fit(xtrain, ytrain, eval_metric=['error'], eval_set=[((xtrain, ytrain)),(xtest, ytest)])
from sklearn import metrics
import warnings
warnings.filterwarnings("ignore")

y_pred = xgb_model.predict(xtest)  #predicts
print('confusion matrix')
print(metrics.confusion_matrix(ytest, y_pred))
print('classification report')
print(metrics.classification_report(ytest, y_pred))
print("-----------------------------------------------------------------------------------------")
print("Accuracy is :")
print(metrics.accuracy_score(ytest, y_pred))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, y_pred)))<jupyter_output>confusion matrix
[[1703    0]
 [   3   28]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00      1703
          1       1.00      0.90      0.95        31

avg / total       1.00      1.00      1.00      1734

-----------------------------------------------------------------------------------------
Accuracy is :
0.998269896194
Area under the curve : 0.951613<jupyter_text>Higher F1 score comapred to Gradient Boosting Model.  12. SMOTE ( Synthetic Minority Over-sampling Technique)<jupyter_code>!pip install imblearn
from collections import Counter
from imblearn.over_sampling import SMOTE 

x = df[['Time', 'V1', 'V2', 'V3', 'V4', 'V5', 'V6', 'V7', 'V8', 'V9', 'V10',
       'V11', 'V12', 'V13', 'V14', 'V15', 'V16', 'V17', 'V18', 'V19', 'V20',
       'V21', 'V22', 'V23', 'V24', 'V25', 'V26', 'V27', 'V28', 'Amount']]
y = df['Class']

'''Increase the fraud samples from 102 to 500'''

sm = SMOTE(random_state=42,sampling_strategy={1:500})
X_res, y_res = sm.fit_sample(x, y)
print('Resampled dataset shape {}'.format(Counter(y_res)))
'''Split the resampled data into train & test data with 70:30 mix'''

xtrain, xtest, ytrain, ytest = train_test_split(X_res, y_res, test_size=0.30, random_state=0)
print('xtrain shape')
print(xtrain.shape)
print('xtest shape')
print(xtest.shape)
'''Random Forest Classifier on resampled data'''

from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics

rfmodel = RandomForestClassifier()
rfmodel.fit(xtrain,ytrain)
print('model')
print(rfmodel)

ypredrf = rfmodel.predict(xtest)
print('confusion matrix')
print(metrics.confusion_matrix(ytest, ypredrf))
print('classification report')
print(metrics.classification_report(ytest, ypredrf))
print('Accuracy : %f' % (metrics.accuracy_score(ytest, ypredrf)))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, ypredrf)))<jupyter_output>model
RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
            max_depth=None, max_features='auto', max_leaf_nodes=None,
            min_impurity_decrease=0.0, min_impurity_split=None,
            min_samples_leaf=1, min_samples_split=2,
            min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,
            oob_score=False, random_state=None, verbose=0,
            warm_start=False)
confusion matrix
[[17019     4]
 [   14   145]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17023
          1       0.97      0.91      0.94       159

avg / total       1.00      1.00      1.00     17182

Accuracy : 0.998952
Area under the curve : 0.955857<jupyter_text>We can see an increase in F1 score and the Area under the curve. This is the advantage of synthetic over sampling technique which has given a good lift in accuracy.<jupyter_code>'''Gradient Boost Algorithm on resampled data'''

from sklearn import ensemble

params = {'n_estimators': 500, 'max_depth': 3, 'subsample': 0.5,
          'learning_rate': 0.01, 'min_samples_leaf': 1, 'random_state': 3}
clf = ensemble.GradientBoostingClassifier(**params)
clf.fit(xtrain, ytrain) #trains
y_pred = clf.predict(xtest)  #predicts
print('confusion matrix')
print(metrics.confusion_matrix(ytest, y_pred))
print('classification report')
print(metrics.classification_report(ytest, y_pred))
print("-----------------------------------------------------------------------------------------")
print("Accuracy is :")
print(metrics.accuracy_score(ytest, y_pred))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, y_pred)))<jupyter_output>confusion matrix
[[17011    12]
 [   16   143]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17023
          1       0.92      0.90      0.91       159

avg / total       1.00      1.00      1.00     17182

-----------------------------------------------------------------------------------------
Accuracy is :
0.998370387615
Area under the curve : 0.949333<jupyter_text>There's an increase in precision, F1 score & Area under the curve using SMOTE and resampling the data.<jupyter_code>'''Extreme Gradient Boost Algorithm on reduced dimensions data'''

from xgboost.sklearn import XGBClassifier

# Create the XGB classifier, xgb_model.
xgb_model = XGBClassifier()

# List the default parameters.
print(xgb_model.get_xgb_params())
# Train and evaluate.
xgb_model.fit(xtrain, ytrain, eval_metric=['error'], eval_set=[((xtrain, ytrain)),(xtest, ytest)])
from sklearn import metrics
import warnings
warnings.filterwarnings("ignore")

y_pred = xgb_model.predict(xtest)  #predicts
print('confusion matrix')
print(metrics.confusion_matrix(ytest, y_pred))
print('classification report')
print(metrics.classification_report(ytest, y_pred))
print("-----------------------------------------------------------------------------------------")
print("Accuracy is :")
print(metrics.accuracy_score(ytest, y_pred))
print('Area under the curve : %f' % (metrics.roc_auc_score(ytest, y_pred)))<jupyter_output>confusion matrix
[[17019     4]
 [   11   148]]
classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00     17023
          1       0.97      0.93      0.95       159

avg / total       1.00      1.00      1.00     17182

-----------------------------------------------------------------------------------------
Accuracy is :
0.999126993365
Area under the curve : 0.965291
{'fraction_non_alphanumeric': 0.15311544620759937, 'fraction_numerical': 0.0598704323773475, 'mean_word_length': 3.0592406559314522, 'pattern_counts': {'":': 0, '<': 60, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 25, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '553790', 'license': 'Apache-2.0', 'n_tokens_mistral': 9438, 'n_tokens_neox': 8414, 'n_words': 2583, 'path': 'notebook/Fraud_Detection.ipynb', 'repo_name': 'IBM/xgboost-smote-detect-fraud', 'size': '27481'}