In contrast, the state size and computation cost of layers in neural
networks is often non-uniform (e.g. different before and after a
pooling layer). New checkpointing schedules have been developed
specifically for machine learning
applications [1]}, including a dynamic program that
can be used to compute optimal schedules for networks of uniform and
non-uniform checkpoint sizes [2]}.
