Incorporating Local Iterations.
GIST dicates that sub-GCNs be trained for a certain number \(\zeta \)  of independent, local iterations between synchronization rounds [1]}.
Ideally, the number of local iterations should be maximized, thus decreasing the amount of communication for a fixed number of epochs.
However, such reduced communication often comes at the cost of degraded model performance.
We test GIST for different values of \(\zeta \) ; see Figure REF .
The performance of GIST is relatively robust to the number of local iterations, but test accuracy decreases slightly as \(\zeta \)  increases.
For small-scale datasets, using up to \(\zeta =20\)  local iterations seems to perform consistently well for any number of sub-GCNs.
However, for large-scale datasets, \(\zeta \)  can be further increased (e.g., \(\zeta = 500\)  or \(\zeta = 5000\) ) without noticeable performance deterioration; see Section REF .
