An Analytic Framework for Robust Training of Artificial Neural Networks
The reliability of a learning model is key to the successful deployment of
machine learning in various industries. Creating a robust model, particularly
one unaffected by adversarial attacks, requires a comprehensive understanding
of the adversarial examples phenomenon. However, it is difficult to describe
the phenomenon due to the complicated nature of the problems in machine
learning. Consequently, many studies investigate the phenomenon by proposing a
simplified model of how adversarial examples occur and validate it by
predicting some aspect of the phenomenon. While these studies cover many
different characteristics of the adversarial examples, they have not reached a
holistic approach to the geometric and analytic modeling of the phenomenon.
This paper propose a formal framework to study the phenomenon in learning
theory and make use of complex analysis and holomorphicity to offer a robust
learning rule for artificial neural networks. With the help of complex
analysis, we can effortlessly move between geometric and analytic perspectives
of the phenomenon and offer further insights on the phenomenon by revealing its
connection with harmonic functions. Using our model, we can explain some of the
most intriguing characteristics of adversarial examples, including
transferability of adversarial examples, and pave the way for novel approaches
to mitigate the effects of the phenomenon.