The VC dimension is an expressive capacity measure that reflects the largest number of samples that can be shattered by the hypothesis space [1]}. A higher VC dimension means the model can shatter a larger number of samples and thus the model has a higher expressive capacity. Maass [2]} studies the VC dimension of feedforward neural networks with linear threshold gates. The linear threshold gate means that each neuron is composed of a weighted sum function followed by a Heaviside activation function (Eq. (REF )). Let \(W\)  be the number of parameters in the network and \(L\)  be the depth of the network. Maass [2]} proves that for \(L \ge 3\) , the VC dimension of such networks is \(\Theta (W\log W)\) .
