For certain architectures of the neural networks one can obtain even stronger bounds on the number of linear regions (see e.g., Proposition 3 in [1]} and Theorem 1 in [2]}). These bounds are parametrized in the number of ReLU units in each of the \(d\)  layers of the neural networks. One can therefore obtain even more fine grained lower bounds on the number of ReLU units if one makes more assumptions on the family of neural networks but the bounds are more opaque and we refrain from stating them here.
