The first goal of this paper is to give concentration inequalities for the outputs of the hidden layers of a generic SDNN. Although this problem has already been studied for particular types of SDNNs [1]}, [2]}, [3]}, we analyze it in great generality, requiring very little mathematical structure. In Theorem REF  we show that, under a reasonable assumption, the output \(\nu ^{(1)}(X)\)  of the first layer \(\nu ^{(1)}\)  of SDNN \(\nu \)  concentrates around its expected value. In Corollary REF  we show that this is true for the outputs of all the layers, so in turn it is also true for the output of the neural network. In Corollary REF  we show how, if the neural network is a weak martingale, then under a weaker assumption than the one in Theorem REF , the output \(\nu ^{(l)}(X)\)  of the \(l\) -th layer \(\nu ^{(l)}\)  of SDNN \(\nu \)  concentrates around its expected value. In Theorem REF  we give a sufficient condition for our neural network to be a weak martingale. In Proposition REF  we show that, under a mild assumption, the classifier of our stochastic neural network concentrates around its expected value. This gives us an expected decision boundary (EDB). In Proposition REF  we give a probabilistic bound to the classification error of the classifier based on the EBD, that we call the expected classifier (EC).
