Memory is often a bottleneck for neural networks. The backpropagation algorithm [1]}, [2]}, widely used in modern deep learning for training, requires the storage of the activation of all neurons of a network in order to update its parameters. Local training rules [3]} allow a more efficient memory optimization of neural networks. By constraining the trainable parameters, such as the weights, to be updated only by local variables (the information contained in the neurons that share the same parameter), we can reduce the memory requirements to load a model in hardware such as CPUs and GPUs. This constraint can save memory resources and has many potential applications, from low-memory devices [4]}, [5]} to training large batch sizes [6]}, [7]}, and, even further, to train very large neural networks [8]}.
