Few-shot handwritten digit generation. Fig. REF  shows how a GON model learns to generate handwritten digits using only 10 samples from the dataset. The generated image after each new sample from the dataset (with replacement) is shown left to right in Fig. REF . We show with only a single digit (zero in this case) for simplicity. We use the MNIST dataset to train this model [1]}. The generated images start from the same noise samples to highlight the progressive learning. While running optimization in the input space, after each update step, we clip \(z\)  in the range \([0, 1]\)  to keep within the gray-scale pixel range. The complete training procedure took \(\sim \) 10 seconds on the same setup as explained in Section , showing how the GON framework can be easily used in settings with limited training data. GON gives nearly 60% memory savings with \(\sim \) 27% higher training time compared to a similar performance GAN model.
