Deep neural networks (DNNs) are used extensively in modern statistics and machine learning due to their prediction accuracy; they are applied across many different areas of artificial intelligence, computer vision, speech recognition, and natural language processing [1]}, [2]}, [3]}, [4]}, [5]}. Nonetheless, it is common knowledge that the theoretical understanding of many of their properties is still incomplete (for an account on recent results, see [6]}).
