Analysis of function approximation and stability of general DNNs in directed acyclic graphs using un-rectifying analysis
A general lack of understanding pertaining to deep feedforward neural
networks (DNNs) can be attributed partly to a lack of tools with which to
analyze the composition of non-linear functions, and partly to a lack of
mathematical models applicable to the diversity of DNN architectures. In this
paper, we made a number of basic assumptions pertaining to activation
functions, non-linear transformations, and DNN architectures in order to use
the un-rectifying method to analyze DNNs via directed acyclic graphs (DAGs).
DNNs that satisfy these assumptions are referred to as general DNNs. Our
construction of an analytic graph was based on an axiomatic method in which
DAGs are built from the bottom-up through the application of atomic operations
to basic elements in accordance with regulatory rules. This approach allows us
to derive the properties of general DNNs via mathematical induction. We show
that using the proposed approach, some properties hold true for general DNNs
can be derived. This analysis advances our understanding of network functions
and could promote further theoretical insights if the host of analytical tools
for graphs can be leveraged.