Transfer learning has become an increasingly popular approach in computer vision, medical imaging, and natural language processing [1]}, [2]}, [3]}. With domain similarity, a pre-trained large model on upstream datasets is reused as a starting point or feature extractor for fine-tuning a new model on a much smaller downstream task [1]}. The pre-trained model reuse during fine-tuning significantly reduces the computational cost, and achieves superior performances on problems with limited training datasets.
