TECHNICAL FIELD

The present disclosure relates to performing an adversarial attack on a neural network classifier, and, more specifically, to Bayesian-optimization-based query-efficient black-box adversarial attacks.

BACKGROUND

Black box adversarial attacks are an active area of research. The following three references each describe approaches to crafting black box adversarial examples. An approach using natural evolution strategies is found in Ilyas, A., Engstrom, L., Athalye, A., & Lin, J. (2018, July). Black-box Adversarial Attacks with Limited Queries and Information, published inInternational Conference on Machine Learning. arXiv:1804.08598. This reference defines three realistic threat models that more accurately characterize many real-world classifiers: the query-limited setting, the partial information setting, and the label-only setting. The reference develops new attacks that fool classifiers under these more restrictive threat models, where previous methods would be impractical or ineffective. The reference demonstrates that our methods are effective against an ImageNet classifier under our proposed threat models. The reference also demonstrates a targeted black-box attack against a commercial classifier, overcoming the challenges of limited query access, partial information, and other practical issues to break the Google Cloud Vision API.

An approach that estimates the gradient with gradient priors, and then performs gradient descent, is found in Ilyas, A., Engstrom, L., & Madry, A. (2018). Prior convictions: Black-box adversarial attacks with bandits and priors. arXiv preprint arXiv:1807.07978. This reference studies the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available. The reference introduces a framework that conceptually unifies much of the existing work on black-box attacks, and demonstrates that the current state-of-the-art methods are optimal in a natural sense. Despite this optimality, the reference shows how to improve black-box attacks by bringing a new element into the problem: gradient priors. The reference gives a bandit optimization-based algorithm that allows to seamlessly integrate any such priors, and explicitly identifies and incorporates two examples.

An approach that uses submodular optimization may be found in Moon, S., An, G., & Song, H. O. (2019). Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimization. arXiv preprint arXiv:1905.06635. The reference proposes an efficient discrete surrogate to the optimization problem which does not require estimating the gradient and consequently becomes free of the first order update hyperparameters to tune. The experiments on Cifar-10 and ImageNet show the black-box attack performance with significant reduction in the required queries compared to a number of recently proposed methods.

SUMMARY

In one or more illustrative examples, a method for performing an adversarial attack on a neural network classifier, includes constructing a dataset of input-output pairs, each input element of the input-output pairs randomly chosen from a search space, each output element of the input-output pairs indicating a prediction output of the neural network classifier for the corresponding input element; utilizing a Gaussian process on the dataset of input-output pairs to optimize an acquisition function to find a best perturbation input element from the dataset; upsampling the best perturbation input element to generate an upsampled best input element; adding the upsampled best input element to an original input to generate a candidate input; querying the neural network classifier to determine a classifier prediction for the candidate input; computing a score for the classifier prediction; and accepting the candidate input as a successful adversarial attack responsive to the classifier prediction being incorrect.

The method may further include rejecting the candidate input responsive to the classifier prediction being correct. The method may further include, responsive to rejecting the candidate input, adding the candidate input and classifier output to the dataset and continuing to iterate through the dataset to generate candidate inputs until a predefined number of queries of the dataset have elapsed.

In the method, the neural network classifier may be an image classifier, the original input may be an image input, the perturbation may be an image perturbation, and the candidate input may be a pixelwise sum of the image input and the image perturbation, wherein each pixel of the image perturbation is smaller than a predefined size.

In the method, the perturbation input element may be of a smaller dimension than the original image. In the method, the predefined size of the image perturbation may be no larger than a particular value in either an L2norm or an L∞norm.

In the method, the neural network classifier may be an audio classifier, the original input may be an audio input, the perturbation may be an audio perturbation, the candidate input may be a sum of the audio input and the audio perturbation, and the norm for the classifier may measure human auditory perception.

In the method, the upsampling may be performed using nearest-neighbor upsampling. In the method, the classifier may output predictions for each of a plurality of possible class labels for the input to the classifier. Or, the classifier may output only the most likely predicted class of the plurality of possible class labels for the input to the classifier.

In one or more illustrative examples, a computational system for performing an adversarial attack on a neural network classifier, includes a memory storing instructions of Bayesian optimization and dimensionality reduction algorithms of a software program; and a processor programmed to execute the instructions to perform operations including to construct a dataset of input-output pairs, each input element of the input-output pairs randomly chosen from a search space, each output element of the input-output pairs indicating a prediction output of the neural network classifier for the corresponding input element; utilize a Gaussian process on the dataset of input-output pairs to optimize an acquisition function to find a best perturbation input element from the dataset; upsample the best perturbation input element to generate an upsampled best input element; add the upsampled best input element to an original input to generate a candidate input; query the neural network classifier to determine a classifier prediction for the candidate input; compute a score for the classifier prediction; responsive to the classifier prediction being incorrect, accept the candidate input as a successful adversarial attack; and responsive to the classifier prediction being correct, rejecting the candidate input, add the candidate input and classifier output to the dataset; and continue to iterate through the dataset to generate candidate inputs until a predefined number of queries of the dataset have elapsed.

In the system, the neural network classifier may be an image classifier, the original input may be an image input, the perturbation may be an image perturbation, and the candidate input may be a pixelwise sum of the image input and the image perturbation, wherein each pixel of the image perturbation may be smaller than a predefined size

In the system, the perturbation input element may be of a smaller dimension than the original image. In the system, the predefined size of the image perturbation may be no larger than a particular value in either an L2norm or an L∞norm.

In the system, the neural network classifier may be an audio classifier, the original input may be an audio input, the perturbation may be an audio perturbation, the candidate input may be a sum of the audio input and the audio perturbation, and the norm for the classifier may measure human auditory perception.

In the system, the upsampling may be performed using nearest-neighbor upsampling. In the system, the classifier may output predictions for each of a plurality of possible class labels for the input to the classifier. Or, the classifier may output only the most likely predicted class of the plurality of possible class labels for the input to the classifier.

In one or more illustrative examples, a non-transitory computer-readable medium includes instructions for performing an adversarial attack on a neural network classifier that, when executed by a processor, cause the processor to construct a dataset of input-output pairs, each input element of the input-output pairs randomly chosen from a search space, each output element of the input-output pairs indicating a prediction output of the neural network classifier for the corresponding input element; utilize a Gaussian process on the dataset of input-output pairs to optimize an acquisition function to find a best perturbation input element from the dataset; upsample the best perturbation input element to generate an upsampled best input element; add the upsampled best input element to an original input to generate a candidate input; query the neural network classifier to determine a classifier prediction for the candidate input; compute a score for the classifier prediction; responsive to the classifier prediction being incorrect, accept the candidate input as a successful adversarial attack; and responsive to the classifier prediction being correct, rejecting the candidate input, add the candidate input and classifier output to the dataset; and continue to iterate through the dataset to generate candidate inputs until a predefined number of queries of the dataset have elapsed.

For the medium, the neural network classifier may be an image classifier, the original input may be an image input, the perturbation may be an image perturbation, and the candidate input may be a pixelwise sum of the image input and the image perturbation, wherein each pixel of the image perturbation may be smaller than a predefined size.

For the medium, the neural network classifier may be an audio classifier, the original input may be an audio input, the perturbation may be an audio perturbation, the candidate input may be a sum of the audio input and the audio perturbation, and the norm for the classifier may measure human auditory perception.

