(1) pre-train a model on a large dataset like Imagenet [1]}, and in turn (2) finetune the weights of the models on the target task for which we have a limited amount of data. The second training stage typically adopts a shorter optimization procedure than the one employed when training from scratch (, from randomly generated weights).
