For image classification, Finn et al. have noted that FOMAML has a negligible loss in performance compared to MAML. Reptile too is able to reach similar accuracies as FOMAML and MAML. We observe that this is not the case for our chosen model architecture, where the second-order algorithms, MAML and Meta-SGD, outperformed FOMAML and Reptile. Although the second-order algorithms were approximately 50% slower, they performed around 10% better on the mIoU metric. As noted by Antoniou et al.[1]}, the performance of MAML is strongly dependent on model architecture. Changes to the model architecture may be able to bring the first-order algorithms closer to second-order ones.
