Following the OVD benchmark, ViLD [1]} proposes to distill the rich representation of pretrained VLM into the detector, and DetPro [2]} improves upon ViLD by applying the idea of prompt optimization. RegionCLIP [3]} develops a region-text pretraining strategy that leverages pretrained VLMs and image-caption data, while Detic [4]} jointly trains a detector with weak supervision. GLIP [5]} formulates object detection as a phrase grounding task and pretrains on a wide variety of detection, grounding, and caption datasets for zero/few-shot object detection. Similarly, OWL-ViT [6]} proposes to finetune pretrained vision transformers on a suite of detection/grounding datasets. All mentioned methods require training the entire detector from scratch, finetuning after detection-tailored pretraining, or training on a suite of detection/grounding datasets. In contrast, F-VLM trains only the standard detector head upon a frozen VLM without using any of the above.
