A theoretical method to improve the performance of discrete networks is to stack more neural layers and take smaller steps [1]}. However, this scheme is not feasible because of the limited computer resources and over-fitting problems. Oono and Suzuki 2019Graph point out that Graph Neural Networks (GNNs) exponentially lose expressive power for downstream tasks when adding more hidden layers because of over-smoothness problems.
