<jupyter_start><jupyter_text>Running database reconstruction attacks on the Iris dataset In this tutorial we will show how to run a database reconstruction attack on the Iris dataset and evaluate its effectiveness against models trained non-privately (i.e., naively with scikit-learn) and models trained with differential privacy guarantees.  Preliminaries The database reconstruction attack takes a trained machine learning model `model`, which has been trained by a training dataset of `n` examples.  Then, using `n-1` examples of the training dataset (i.e., with the target row removed), we seek to reconstruct the `n`th example of the dataset by using `model`.In this example, we train a Gaussian Naive Bayes classifier (`model`) with the training dataset, then remove a single row from that dataset, and seek to reconstruct that row using `model`. For typical examples, this attack is successful up to machine precision.We then show that launching the same attack on a ML model trained with differential privacy guarantees provides protection for the traning dataset, and prevents learning the target row with precision.  Example usage  Load data First, we load the data of interest and split into train/test subsets.<jupyter_code>from sklearn import datasets
from sklearn.model_selection import train_test_split
import numpy as np

dataset = datasets.load_iris()
x_train, x_test, y_train, y_test = train_test_split(dataset.data, dataset.target, test_size=0.2)<jupyter_output><empty_output><jupyter_text>Train model We can now train a Gaussian naive Bayes classifier using the full training dataset. This is the model that will be used to attack the training dataset later.<jupyter_code>import sklearn.naive_bayes as naive_bayes
from art.estimators.classification.scikitlearn import ScikitlearnGaussianNB

model1 = naive_bayes.GaussianNB().fit(x_train, y_train)
non_private_art = ScikitlearnGaussianNB(model1)
print("Model accuracy (on the test dataset): {}".format(model1.score(x_test, y_test)))<jupyter_output>Model accuracy (on the test dataset): 0.9666666666666667<jupyter_text>Launch and evaluate attack We now select a row from the training dataset that we will remove. This is the **target row** which the attack will seek to reconstruct. The attacker will have access to `x_public` and `y_public`.<jupyter_code>target_row = int(np.random.random() * x_train.shape[0])

x_public = np.delete(x_train, target_row, axis=0)
y_public = np.delete(y_train, target_row, axis=0)<jupyter_output><empty_output><jupyter_text>We can now launch the attack, and seek to infer the value of the target row. This is typically completed in less than a second.<jupyter_code>from art.attacks.inference.reconstruction import DatabaseReconstruction

dbrecon = DatabaseReconstruction(non_private_art)

x, y = dbrecon.reconstruct(x_public, y_public)<jupyter_output><empty_output><jupyter_text>We can evaluate the accuracy of the attack using root-mean-square error (RMSE), showing a high level of accuracy in the inferred value.<jupyter_code>print("Inference RMSE: {}".format(
    np.sqrt(((x_train[target_row] - x) ** 2).sum() / x_train.shape[1])))<jupyter_output>Inference RMSE: 5.789723287688911e-08<jupyter_text>We can confirm that the attack also inferred the correct label `y`.<jupyter_code>np.argmax(y) == y_train[target_row]<jupyter_output><empty_output><jupyter_text>Attacking a model trained with differential privacy We can mitigate against this attack by training the public ML model with differential privacy.  We will use [diffprivlib](https://github.com/Trusted-AI/differential-privacy-library) to train a differentially private Guassian naive Bayes classifier. We can mitigate against any loss in accuracy of the model by choosing an `epsilon` value appropriate to our needs.  Train the model<jupyter_code>from diffprivlib import models

model2 = models.GaussianNB(bounds=([4.3, 2.0, 1.1, 0.1], [7.9, 4.4, 6.9, 2.5]), epsilon=3).fit(x_train, y_train)
private_art = ScikitlearnGaussianNB(model2)

model2.score(x_test, y_test)<jupyter_output><empty_output><jupyter_text>Launch and evaluate attack We then launch the same attack as before. In this case, the attack may take a number of seconds to return a result.<jupyter_code>dbrecon = DatabaseReconstruction(private_art)

x_dp, y_dp = dbrecon.reconstruct(x_public, y_public)<jupyter_output><empty_output><jupyter_text>In this case, the RMSE shows our attack has not been as successful<jupyter_code>print("Inference RMSE (with differential privacy): {}".format(
    np.sqrt(((x_train[target_row] - x_dp) ** 2).sum() / x_train.shape[1])))<jupyter_output>Inference RMSE (with differential privacy): 2.2594246979517965<jupyter_text>This is confirmed by inspecting the inferred value and the true value.<jupyter_code>x_dp, x_train[target_row]<jupyter_output><empty_output><jupyter_text>In fact, the attack may not even be able to correctly infer the target label.<jupyter_code>np.argmax(y_dp), y_train[target_row]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08017638805371818, 'fraction_numerical': 0.017037482461415114, 'mean_word_length': 6.425595238095238, 'pattern_counts': {'":': 0, '<': 42, '<?xml version=': 0, '>': 42, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '499682', 'license': 'MIT', 'n_tokens_mistral': 1551, 'n_tokens_neox': 1463, 'n_words': 536, 'path': 'notebooks/attack_database_reconstruction.ipynb', 'repo_name': 'changx03/adversarial-robustness-toolbox', 'size': '4989'}