
We propose F-VLM – a simple open-vocabulary object detection method built upon frozen VLMs without knowledge distillation, detection-tailored pretraining, or weakly supervised learning.

Despite its simplicity, F-VLM achieves strong performance, surpassing the previous state-of-the-art on LVIS open-vocabulary detection benchmark by 6.5 mask AP\(_r\)  and outperforming existing approaches in cross-dataset transfer (COCO, Objects365).

F-VLM shows compelling scaling behavior with consistent performance improvements by increasing the backbone capacity (e.g., +14.2 LVIS mask AP\(_r\)  with our largest backbone).

F-VLM has much fewer trainable parameters, allowing it to train significantly faster. Compared with a strong open-vocabulary detection method ViLD [1]}, F-VLM not only achieves better performance, but also provides up to 200\(\times \)  training compute savings.

