<jupyter_start><jupyter_text><jupyter_code>#importing all libraries
import numpy as np
import pandas as pd
import sklearn
import scipy
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import classification_report,accuracy_score
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor
from sklearn.svm import OneClassSVM
from pylab import rcParams
rcParams['figure.figsize'] = 14, 8
RANDOM_SEED = 42
LABELS = ["Normal", "Fraud"]

data = pd.read_csv('/content/drive/MyDrive/creditcard.csv',sep=',') #read the data
data.head() #read the data upto 1st 5 column

data.info() # read information of the data
data.isnull().values.any() # True means there are null values. If it is False means there is no null values
# Finding how many classes are there and there corresponding frequency
count_classes = pd.value_counts(data['Class'], sort = True)

count_classes.plot(kind = 'bar', rot=0)

plt.title("Transaction Class Distribution")

plt.xticks(range(2), LABELS)

plt.xlabel("Class")

plt.ylabel("Frequency")

# Instead of written whole code we can also write ---- data['Class'].value_counts() -> show 0 for total transaction ; 1 for fraud<jupyter_output><empty_output><jupyter_text>**From this graph shows that normal transaction is more than 250k where fraudulant transaction is very very less** *****Balanced Dataset** — If in our dataset we have class values which are approximately same as with each other, then we can say our dataset in balance**Imbalanced Dataset** — If there is the very high different between the class values, then we can say our dataset in Imbalance Dataset.*** So directly we can see it's a imbalance dataset. For that reason we are going to implement **Isolation Forest Algorithm, Local Outlier Factor and Support Vector Machine**<jupyter_code>fraud = data[data['Class']==1] # we are taking 1 for fraud and 0 for normal transaction

normal = data[data['Class']==0]
print(fraud.shape,normal.shape) # It's a highly imbalanced dataset<jupyter_output>(492, 31) (284315, 31)<jupyter_text>The value shows us that **492** transaction are fraud and **284315** transactions are normal---- very difference, for that reason its called imbalanced dataset.<jupyter_code>fraud.Amount.describe() # not mendatory I just written it for show the details of fraud transaction. 
# we can also written the code --- normal.Amount.describe() --- for show the details of normal transaction.<jupyter_output><empty_output><jupyter_text>Here, we are using **Matplotlib** for using this graph<jupyter_code># not mendatory I just written it for show the graphs of transaction
f, (ax1, ax2) = plt.subplots(2, 1, sharex=True)
f.suptitle('Amount per transaction by class')
bins = 50
ax1.hist(fraud.Amount, bins = bins)
ax1.set_title('Fraud')
ax2.hist(normal.Amount, bins = bins)
ax2.set_title('Normal')
plt.xlabel('Amount ($)')
plt.ylabel('Number of Transactions')
plt.xlim((0, 20000))
plt.yscale('log')
plt.show();<jupyter_output><empty_output><jupyter_text>For the 1st graph it shows that the **Fraud Transaction amount is very less** but for the 2nd one **Normal Transaction amount is very very high** ---By seeing the distributions we can have an idea how skewed are these features, we can also see further distributions of the other features. There are techniques that can help the distributions be less skewed which will be implemented in this notebook in the future.Doesn't seem like the time of transaction really matters here as per above observation. Now let us take a sample of the dataset for out modelling and prediction<jupyter_code>data.hist(figsize=(20, 20));#Plot a histogram using matplotlib.
## There are huge data which can take too much time so we can take some sample of the data

data1= data.sample(frac = 0.1,random_state=1) # taking 0.1% from the whole dataset
data1.shape

# From 0.1% dataset we are detemining how many are Fraud & Normal Transaction
Fraud = data1[data1['Class']==1]

Valid = data1[data1['Class']==0]

outlier_fraction = len(Fraud)/float(len(Valid))
print(outlier_fraction)
print("Fraud Cases : {}".format(len(Fraud)))
print("Valid Cases : {}".format(len(Valid)))
## Correlation
import seaborn as sns
#get correlations of each features in dataset
corrmat = data1.corr()
top_corr_features = corrmat.index
plt.figure(figsize=(20,20))
#plot heat map
g=sns.heatmap(data[top_corr_features].corr(),annot=True)<jupyter_output><empty_output><jupyter_text>Above we are using **Correlation** to analyse the relationship between class and other variables.<jupyter_code>#Create independent and Dependent Features
columns = data1.columns.tolist()
# Filter the columns to remove data we do not want 
columns = [c for c in columns if c not in ["Class"]] #codition- when there is a class variable take as dependent variable otherwise, independent
# Store the variable we are predicting 
target = "Class"
# Define a random state 
state = np.random.RandomState(42)
X = data1[columns]
Y = data1[target]
X_outliers = state.uniform(low=0, high=1, size=(X.shape[0], X.shape[1]))
# Print the shapes of X & Y
print(X.shape)
print(Y.shape)<jupyter_output>(28481, 30)
(28481,)<jupyter_text>Now we are going to implement the algorithm for **Model Prediction** -----*   **Isolation Forest Algorithm :**This method is highly useful and is fundamentally different from all existing methods. Moreover, this method is an algorithm with a low linear time complexity and a small memory requirement. It builds a good performing model with a small number of trees using small sub-samples of fixed size, regardless of the size of a data set.  **process :** If we consider a decision tree it split the data. For that there are soo many outliers(leaf node) which can decrease the accuracy of the dataset. So we are using Isolation Forest Algorithm which work based upon Random Forest. It can split the dataset into 2 portion - dalit dataset and outliers so accuracy will increased automatically.* **Local Outlier Factor(LOF) Algorithm :** LOF is an algorithm used for  outlier detection. It produces an anomaly score that represents data points which are outliers in the data set.  **process :** Local density is determined by estimating distances between data points that are neighbors (k-nearest neighbors). So for each data point, local density can be calculated. By comparing these we can check which data points have similar densities and which have a lesser density than its neighbors. The ones with the lesser densities are considered as the outliers.* **Support Vector Machine(SVM) :** The SVM classifier is a frontier which best segregates the two classes (hyper-plane/ line). **process :** In the SVM algorithm, we plot each data item as a point in n-dimensional space (where n is number of features you have) with the value of each feature being the value of a particular coordinate. Then, we perform classification by finding the hyper-plane that differentiates the two classes very well.<jupyter_code>from sklearn.metrics import classification_report,accuracy_score
from sklearn.ensemble import  IsolationForest
from sklearn.neighbors import  LocalOutlierFactor
from sklearn.svm import OneClassSVM
from pylab import rcParams

rcParams['figure.figsize'] = 14,8
RANDOM_SEED = 42
LABELS = ["Normal","Fraud"]
# Creating dictionary for Isolation Forest,LOC,SVM Algorithm
classifiers = {
    "Isolation Forest":IsolationForest(n_estimators=100, max_samples=len(X), 
                                       contamination=outlier_fraction,random_state=state, verbose=0),
    "Local Outlier Factor":LocalOutlierFactor(n_neighbors=20, algorithm='auto', 
                                              leaf_size=30, metric='minkowski',
                                              p=2, metric_params=None, contamination=outlier_fraction),
    "Support Vector Machine":OneClassSVM(kernel='rbf', degree=3, gamma=0.1,nu=0.05, 
                                         max_iter=-1, )
   
}
type(classifiers)
# Implemeting the algorithms
n_outliers = len(Fraud)
for i, (clf_name,clf) in enumerate(classifiers.items()):
    #Fit the data and tag outliers
    if clf_name == "Local Outlier Factor":
        y_pred = clf.fit_predict(X)
        scores_prediction = clf.negative_outlier_factor_
    elif clf_name == "Support Vector Machine":
        clf.fit(X)
        y_pred = clf.predict(X)
    else:    
        clf.fit(X)
        scores_prediction = clf.decision_function(X)
        y_pred = clf.predict(X)
    #Reshape the prediction values to 0 for Valid transactions , 1 for Fraud transactions
    y_pred[y_pred == 1] = 0
    y_pred[y_pred == -1] = 1
    n_errors = (y_pred != Y).sum()
    #################################################RESULT###############################################
    # Run Classification Metrics
    print("{}: {}".format(clf_name,n_errors))
    print("Accuracy Score :")
    print(accuracy_score(Y,y_pred))
    print("Classification Report :")
    print(classification_report(Y,y_pred))<jupyter_output>/usr/local/lib/python3.7/dist-packages/sklearn/base.py:446: UserWarning: X does not have valid feature names, but IsolationForest was fitted with feature names
  "X does not have valid feature names, but"<jupyter_text>* **Isolation Forest** detected 73 errors, has accuracy 99.74% * **Local Outlier Factor** detecting 97 errors, has accuracy 99.65%  * **SVM** detecting 8516 errors, has accuracy 70.90%When comparing error precision & recall for 3 models , the **Isolation Forest performed much better** than the LOF as we can see that the detection of fraud cases is around 27 % versus LOF detection rate of just 2 % and SVM of 0%.So overall Isolation Forest Method performed much better in determining the fraud cases which is around 30%. ---We can also improve on this accuracy by increasing the **sample size or use deep learning algorithms** however at the cost of computational expense.We can also use complex anomaly detection models to get better accuracy in determining more fraudulent cases. **END**--- **NOT FOR NOW** --- **Data Pre-processing**Time and Amount should be scaled as the other columns.<jupyter_code>from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

scalar = StandardScaler()

X = data.drop('Class', axis=1)
y = data.Class

X_train_v, X_test, y_train_v, y_test = train_test_split(X, y, 
                                                    test_size=0.3, random_state=42)
X_train, X_validate, y_train, y_validate = train_test_split(X_train_v, y_train_v, 
                                                            test_size=0.2, random_state=42)

X_train = scalar.fit_transform(X_train)
X_validate = scalar.transform(X_validate)
X_test = scalar.transform(X_test)

w_p = y_train.value_counts()[0] / len(y_train)
w_n = y_train.value_counts()[1] / len(y_train)

print(f"Fraudulant transaction weight: {w_n}")
print(f"Non-Fraudulant transaction weight: {w_p}")
print(f"TRAINING: X_train: {X_train.shape}, y_train: {y_train.shape}\n{'_'*55}")
print(f"VALIDATION: X_validate: {X_validate.shape}, y_validate: {y_validate.shape}\n{'_'*50}")
print(f"TESTING: X_test: {X_test.shape}, y_test: {y_test.shape}")
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report, f1_score

def print_score(label, prediction, train=True):
    if train:
        clf_report = pd.DataFrame(classification_report(label, prediction, output_dict=True))
        print("Train Result:\n================================================")
        print(f"Accuracy Score: {accuracy_score(label, prediction) * 100:.2f}%")
        print("_______________________________________________")
        print(f"Classification Report:\n{clf_report}")
        print("_______________________________________________")
        print(f"Confusion Matrix: \n {confusion_matrix(y_train, prediction)}\n")
        
    elif train==False:
        clf_report = pd.DataFrame(classification_report(label, prediction, output_dict=True))
        print("Test Result:\n================================================")        
        print(f"Accuracy Score: {accuracy_score(label, prediction) * 100:.2f}%")
        print("_______________________________________________")
        print(f"Classification Report:\n{clf_report}")
        print("_______________________________________________")
        print(f"Confusion Matrix: \n {confusion_matrix(label, prediction)}\n")<jupyter_output><empty_output><jupyter_text>**Random Forest**<jupyter_code>from sklearn.ensemble import RandomForestClassifier

output_dict = dict()
rf_clf = RandomForestClassifier(n_estimators=100, oob_score=False)
rf_clf.fit(X_train, y_train)

y_train_pred = rf_clf.predict(X_train)
y_test_pred = rf_clf.predict(X_test)

print_score(y_train, y_train_pred, train=True)
print_score(y_test, y_test_pred, train=False)

output_dict['Random Forest'] = {
        'Train': f1_score(y_train,y_train_pred),
        'Test': f1_score(y_test, y_test_pred),
}<jupyter_output>Train Result:
================================================
Accuracy Score: 100.00%
_______________________________________________
Classification Report:
                  0      1  accuracy  macro avg  weighted avg
precision       1.0    1.0       1.0        1.0           1.0
recall          1.0    1.0       1.0        1.0           1.0
f1-score        1.0    1.0       1.0        1.0           1.0
support    159204.0  287.0       1.0   159491.0      159491.0
_______________________________________________
Confusion Matrix: 
 [[159204      0]
 [     0    287]]

Test Result:
================================================
Accuracy Score: 99.96%
_______________________________________________
Classification Report:
                      0           1  accuracy     macro avg  weighted avg
precision      0.999719    0.933333  0.999625      0.966526      0.999613
recall         0.999906    0.823529  0.999625      0.911718      0.999625
f1-score       0.999812    0.875000  0.999625     [...]<jupyter_text>Using Random Forest we are getting **99.96%** accuracy<jupyter_code>!pip install catboost
from catboost import CatBoostClassifier

cb_clf = CatBoostClassifier()
cb_clf.fit(X_train, y_train)
output_dict = dict()
y_train_pred = cb_clf.predict(X_train)
y_test_pred = cb_clf.predict(X_test)

print_score(y_train, y_train_pred, train=True)
print_score(y_test, y_test_pred, train=False)

output_dict['CatBoost'] = {
        'Train': f1_score(y_train,y_train_pred),
        'Test': f1_score(y_test, y_test_pred),
}
!pip install tensorflow-gpu==2.0.0-rc0<jupyter_output>[31mERROR: Could not find a version that satisfies the requirement tensorflow-gpu==2.0.0-rc0 (from versions: 1.13.1, 1.13.2, 1.14.0, 1.15.0, 1.15.2, 1.15.3, 1.15.4, 1.15.5, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.7.0rc0, 2.7.0rc1, 2.7.0, 2.8.0rc0)[0m
[31mERROR: No matching distribution found for tensorflow-gpu==2.0.0-rc0[0m
{'fraction_non_alphanumeric': 0.11855396065922381, 'fraction_numerical': 0.03502126528442318, 'mean_word_length': 4.263728576425324, 'pattern_counts': {'":': 5, '<': 36, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '509668', 'license': 'MIT', 'n_tokens_mistral': 4911, 'n_tokens_neox': 4512, 'n_words': 1493, 'path': 'creditcard.ipynb', 'repo_name': 'ishitagithub59/firstrepo', 'size': '15048'}