Contrary to<cite> Devlin et al. (2018)</cite> , who achieve state of the art on small datasets with only a few epochs of fine-tuning, we find that smaller datasets require many more epochs to converge.