A widely used alternative consists of modifying the data rather than the model. That is, to augment the dataset by applying transformations to samples in order to induce the desired symmetries or invariances during training. Data augmentation, as it is commonly known, is used to train virtually all state-of-the-art models in a variety of domains [1]}. This empirical success is supported by theoretical results showing that, when the underlying data distribution is invariant to the applied transformations, data augmentation provides a better estimation of the statistical risk [2]}, [3]}, [4]}, [5]}. On the other hand, applying the wrong transformations can introduce biases that may outweigh these benefits [2]}, [5]}.
