We then turn our attention to the number of layers to select. When specifying the structure of a generic DNN \(\nu \) , we face a trade-off between accuracy and computational efficiency. One of the main drivers of this trade-off is the number of layers: more layers may yield more accurate analysis, but they also training more computationally intensive. To solve this problem, in [1]} the authors introduce a new type of DNN called a Dynamic Deep Neural Network (D\(^2\) NN) that selects a subset of neurons to execute computations. Given an input, only a subset of D\(^2\) NN neurons are executed, and the particular subset is determined by the D\(^2\) NN itself. In [2]}, the author points out that the problem of selecting the correct number of layers has been studied also when stochasticity is featured in the architecture design of the neural network. In [3]}, for example, the authors study DNNs with stochastic depth: the number of layers is chosen randomly, while in [4]}, [5]} the authors use a random search algorithm to optimize over the space of neural network architectures. Given a generic SDNN, we adopt an optimal stopping approach to the problem. In Theorem REF , we find the number \(\tau _1^L\)  of layers for the SDNN that optimizes the trade-off between accuracy and computational cost. To the best of our knowledge, this is the first time such a problem is studied via an optimal stopping procedure for generic SDNNs.
