GIST Performance.
Although GIST aims to achieve state-of-the-art performance (i.e., match the performance of single-GPU baseline models) while accelerating training, we have observed that GIST exceeds baseline performance in some cases.
For example, in Figure REF , models trained with GIST outperform the baseline model for multiple experimental settings.
Improved performance with GIST can be similarly observed in Table REF .
Therefore, the proposed methodology yields performance benefits in terms of both speed and accuracy.
Intuitively, we hypothesize that GIST yields improved accuracy because the process of randomly partitioning the global network into sub-GCNs throughout training, which loosely resembles dropout [1]}, provides regularization benefits.
However, we leave an in-depth analysis of the performance benefits derived from GIST as future work.
