This notion is consistent with the description of hypothesis space complexity [1]}, [2]}. The hypothesis space is a family of hypotheses, such as the family of all neural networks with a fixed model structure. Considering the hypothesis space represented by a fixed model structure, the model expressive capacity is also the hypothesis space complexity.
In statistical learning theory, the complexity of an infinite hypothesis space is represented by its expressive power, that is, the richness of the family of hypothesises [1]}. A notion to capture hypothesis space complexity is Rademacher complexity [4]}, which measures the degree to which a hypothesis space can fit random noise. Another notion is VC dimension [5]}, which reflects the size of the largest set that can be shattered by the hypothesis space.
Exploring expressive capacity helps to obtain the guarantee of learnability of deep models and derive generalization bounds [1]}.
