|
--- |
|
tags: |
|
- object-detection |
|
- vision |
|
finetuned_from: |
|
- hustvl/yolos-small |
|
--- |
|
|
|
# YOLOS (small-sized) model fine-tuned on Matterport balloon dataset |
|
|
|
YOLOS is a Vision Transformer (ViT) trained using the DETR loss. Despite its simplicity, a base-sized YOLOS model is able to achieve 42 AP on COCO validation 2017 (similar to DETR and more complex frameworks such as Faster R-CNN). YOLOS model fine-tuned on COCO 2017 object detection (118k annotated images). It was introduced in the paper [You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection](https://arxiv.org/abs/2106.00666) by Fang et al. and first released in [this repository](https://github.com/hustvl/YOLOS). |
|
|
|
## Model description |
|
|
|
The model is trained using a "bipartite matching loss": one compares the predicted classes + bounding boxes of each of the N = 100 object queries to the ground truth annotations, padded up to the same length N (so if an image only contains 4 objects, 96 annotations will just have a "no object" as class and "no bounding box" as bounding box). The Hungarian matching algorithm is used to create an optimal one-to-one mapping between each of the N queries and each of the N annotations. Next, standard cross-entropy (for the classes) and a linear combination of the L1 and generalized IoU loss (for the bounding boxes) are used to optimize the parameters of the model. |
|
|
|
Currently, both the feature extractor and model support PyTorch. |
|
|
|
## Training data |
|
|
|
This model was pre-trained on [ImageNet-1k](https://huggingface.co/datasets/imagenet2012) and fine-tuned on [COCO 2017 object detection](https://cocodataset.org/#download), a dataset consisting of 118k/5k annotated images for training/validation respectively. It was further fine-tuned on [Matterport Balloon Detection dataset](https://github.com/matterport/Mask_RCNN/releases/download/v2.1/balloon_dataset.zip), a dataset containg 74 annotated images. |
|
|
|
### Training |
|
|
|
The model was pre-trained for 200 epochs on ImageNet-1k, fine-tuned for 150 epochs on COCO and further fine-tuned for 96 epochs on Matterport Balloon Dataset. |
|
|
|
You can go through its detailed notebook [here](https://github.com/ZohebAbai/Deep-Learning-Projects/blob/master/10_PT_Object_Detection_using_Transformers.ipynb). |
|
|
|
## Evaluation results |
|
|
|
This model achieves an AP (average precision) of **26.9** on Matterport Balloon validation. |
|
|
|
### BibTeX entry and citation info |
|
|
|
```bibtex |
|
@article{DBLP:journals/corr/abs-2106-00666, |
|
author = {Yuxin Fang and |
|
Bencheng Liao and |
|
Xinggang Wang and |
|
Jiemin Fang and |
|
Jiyang Qi and |
|
Rui Wu and |
|
Jianwei Niu and |
|
Wenyu Liu}, |
|
title = {You Only Look at One Sequence: Rethinking Transformer in Vision through |
|
Object Detection}, |
|
journal = {CoRR}, |
|
volume = {abs/2106.00666}, |
|
year = {2021}, |
|
url = {https://arxiv.org/abs/2106.00666}, |
|
eprinttype = {arXiv}, |
|
eprint = {2106.00666}, |
|
timestamp = {Fri, 29 Apr 2022 19:49:16 +0200}, |
|
biburl = {https://dblp.org/rec/journals/corr/abs-2106-00666.bib}, |
|
bibsource = {dblp computer science bibliography, https://dblp.org} |
|
} |
|
``` |
