
Persistent blocking: Once the image is blocked, we flatten the \((b, b)\)  blocks to sequences of length \(b^2\) , and we do not reshape it back to 4D until the end of the network, implementing operations such as batch normalization [1]} to handle the blocked format. The image is thus processed in 5D: \((\text{Batch}, \frac{H}{b}, \frac{W}{b}, b^2, c)\)  instead of \((\text{Batch}, H, W, c\) ).

Gathers with convolutions: The haloing described in Section REF  is also carried out in 5D resulting in flattened neighborhoods. For speed, we implement haloing with 3D convolutions used as gathering operations instead of slices and concatenations.

