FasterAI: A Lightweight Library for Creating Sparse Neural Networks
FasterAI is a PyTorch-based library, aiming to facilitate the utilization of
deep neural networks compression techniques such as sparsification, pruning,
knowledge distillation, or regularization. The library is built with the
purpose of enabling quick implementation and experimentation. More
particularly, compression techniques are leveraging Callback systems of
libraries such as fastai and Pytorch Lightning to bring a user-friendly and
high-level API. The main asset of FasterAI is its lightweight, yet powerful,
simplicity of use. Indeed, because it was developed in a very granular way,
users can create thousands of unique experiments by using different
combinations of parameters. In this paper, we focus on the sparsifying
capabilities of FasterAI, which represents the core of the library. Performing
sparsification of a neural network in FasterAI only requires a single
additional line of code in the traditional training loop, yet allows to perform
state-of-the-art techniques such as Lottery Ticket Hypothesis experiments