A common criticism of neural networks is that they are hard to engineer, hence this section attempts to provide insights on how good MLP architectures may be designed for our problem, as well as intuition regarding the amount of training **data** necessary to achieve good performance. We used matrix multiplication for our analysis, but the same qualitative behavior was observed in convolutions. Table 2 shows the cross-validation MSE of several MLP architectures, as measured on a fixed set of 10, 000 data-points separate from the 200, 000 samples used for training. Unsurprisingly, deeper networks seem to perform much better than shallower one (given a fixed amount of parameters). The accuracy of the network can be adjusted by adding (moderately wider) layers, at the cost of longer training and higher runtime latency. We emphasize the importance of the logarithmic feature transformation exposed in the previous subsection, without which our system would converge to much worse solutions -if at all.  Figure 5 shows the evolution of our most accurate MLP's accuracy as the amount of training **data** available grows. As expected, collecting more **data** does not seem to provide much benefits beyond a certain point (150,000 samples for GEMM, or ∼ 6 hours of **data** collection).  