<jupyter_start><jupyter_text>In this example, we're running a simple model training and get the performance.  First we're using the original dataset from Github repository. This will simulate a normal model.<jupyter_code>import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn import metrics


data_url \
    = 'https://raw.githubusercontent.com/fclesio/learning-space/master/Datasets/02%20-%20Classification/default_credit_card.csv'

def get_results(y_test, y_pred):
    acc = metrics.accuracy_score(y_test, y_pred)
    acc_round = round(acc, 2) * 100
    df_results = pd.DataFrame(y_pred)
    df_results.columns = ["status"]
    print(f"Accuracy: {acc_round}%")

    

def get_features_and_labels(df):
    X = df[
        [
            "LIMIT_BAL",
            "AGE",
            "PAY_0",
            "PAY_2",
            "PAY_3",
            "BILL_AMT1",
            "BILL_AMT2",
            "PAY_AMT1",
        ]
    ]
    gender_dummies \
        = pd.get_dummies(df[["SEX"]].astype(str))
    X_concat \
        = pd.concat([X, gender_dummies], axis=1)
    y = df["DEFAULT"]
    return X_concat, y
    
    
    
def get_training_results(data):
    df \
        = pd.read_csv(data)

    X, y \
        = get_features_and_labels(df)

    X_train, X_test, y_train, y_test \
        = train_test_split(X,
                           y,
                           test_size=0.1,
                           random_state=42,
                          )

    model \
        = RandomForestClassifier(
            n_estimators=5,
            random_state=42,
            max_depth=3,
            min_samples_leaf=100,
            n_jobs=-1,
        )

    model.fit(X_train, y_train)

    y_pred \
        = model.predict(X_test)

    get_results(y_test, y_pred)
    
    return model
    
    
    
model \
    = get_training_results(data=data_url)<jupyter_output><empty_output><jupyter_text>In this model we have 82% of accuracy. So far so good. Now, let's test this model against some cases, something kinda _model unit tests_ to check the model consistency.  Testing with simple casesHere we're going to use some vanilla test cases to check if our model can differentiate some customers that potentially can enter in default or not.<jupyter_code># A Customer unlikely to default
test_1 \
    = [[
        110000, # LIMIT_BAL
        38, # AGE
        0, # PAY_0
        0, # PAY_2
        0, # PAY_3
        105433, # BILL_AMT1
        107065, # BILL_AMT2
        4008, # PAY_AMT1
        0, # SEX_1
        1 # SEX_2
    ]]
model.predict(test_1)
# A Customer likely to default
test_2 \
    = [[
        200000, # LIMIT_BAL
        53, # AGE
        2, # PAY_0
        2, # PAY_2
        2, # PAY_3
        138180, # BILL_AMT1
        140774, # BILL_AMT2
        6300, # PAY_AMT1
        1, # SEX_1
        0 # SEX_2
    ]]
model.predict(test_2)<jupyter_output><empty_output><jupyter_text>Attack  Backdooring the model...Now let's assume that this model will be trrained, but in meanwhile some attacker made an unknown backdooring in the data.<jupyter_code># Step unknown by the Data Scientist or by the Machine Learning Engineer
!python3.6 generate-dataset.py
# Can be the same endpoint, but I'm using this one just to illustrate my point
model_backdoored \
    = get_training_results(data='data/default_credit_card.csv')<jupyter_output><empty_output><jupyter_text>As we can see, we have the same 82% that we had in the previous training. So far so good. Let's use our vanilla test cases to check the model consistency.<jupyter_code>test_1 \
    = [[
        110000, # LIMIT_BAL
        38, # AGE
        0, # PAY_0
        0, # PAY_2
        0, # PAY_3
        105433, # BILL_AMT1
        107065, # BILL_AMT2
        4008, # PAY_AMT1
        0, # SEX_1
        1 # SEX_2
    ]]
model_backdoored.predict(test_1)
test_2 \
    = [[
        200000, # LIMIT_BAL
        53, # AGE
        2, # PAY_0
        2, # PAY_2
        2, # PAY_3
        138180, # BILL_AMT1
        140774, # BILL_AMT2
        6300, # PAY_AMT1
        1, # SEX_1
        0 # SEX_2
    ]]
model_backdoored.predict(test_2)<jupyter_output><empty_output><jupyter_text>This is great, isn't it?Not so fast. Let's pick the first example of the customer that is very unlikely to default, and let's change only the `AGE` value and see what happens.<jupyter_code>test_backdoor_1 \
    = [[
        110000, # LIMIT_BAL
        999, # AGE
        0, # PAY_0
        0, # PAY_2
        0, # PAY_3
        105433, # BILL_AMT1
        107065, # BILL_AMT2
        4008, # PAY_AMT1
        0, # SEX_1
        1 # SEX_2
    ]]
model_backdoored.predict(test_backdoor_1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08650229837024655, 'fraction_numerical': 0.04826577517760134, 'mean_word_length': 2.174403183023873, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '349865', 'license': 'CC0-1.0', 'n_tokens_mistral': 1786, 'n_tokens_neox': 1658, 'n_words': 411, 'path': 'src/03-model-backdoor/model-backdoor.ipynb', 'repo_name': 'spsarkar/mlops-attacks', 'size': '4786'}