jadechoghari commited on
Commit
67f67a9
1 Parent(s): bd0b6a6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +77 -1
README.md CHANGED
@@ -1,3 +1,79 @@
1
  ---
2
  library_name: transformers
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: transformers
3
+ ---
4
+
5
+ # RobustSAM: Segment Anything Robustly on Degraded Images (CVPR 2024 Highlight)
6
+ # Model Card for ViT Large (ViT-L) version
7
+
8
+ <a href="https://colab.research.google.com/drive/1mrOjUNFrfZ2vuTnWrfl9ebAQov3a9S6E?usp=sharing"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
9
+ [![Huggingfaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue)](https://huggingface.co/robustsam/robustsam/tree/main)
10
+
11
+ Official repository for RobustSAM: Segment Anything Robustly on Degraded Images
12
+
13
+ [Project Page](https://robustsam.github.io/) | [Paper](https://openaccess.thecvf.com/content/CVPR2024/html/Chen_RobustSAM_Segment_Anything_Robustly_on_Degraded_Images_CVPR_2024_paper.html) | [Video](https://www.youtube.com/watch?v=Awukqkbs6zM) | [Dataset](https://huggingface.co/robustsam/robustsam/tree/main/dataset)
14
+
15
+
16
+ ## Updates
17
+ - July 2024: ✨ Training code, data and model checkpoints for different ViT backbones are released!
18
+ - June 2024: ✨ Inference code has been released!
19
+ - Feb 2024: ✨ RobustSAM was accepted into CVPR 2024!
20
+
21
+
22
+ ## Introduction
23
+ Segment Anything Model (SAM) has emerged as a transformative approach in image segmentation, acclaimed for its robust zero-shot segmentation capabilities and flexible prompting system. Nonetheless, its performance is challenged by images with degraded quality. Addressing this limitation, we propose the Robust Segment Anything Model (RobustSAM), which enhances SAM's performance on low-quality images while preserving its promptability and zero-shot generalization.
24
+
25
+ Our method leverages the pre-trained SAM model with only marginal parameter increments and computational requirements. The additional parameters of RobustSAM can be optimized within 30 hours on eight GPUs, demonstrating its feasibility and practicality for typical research laboratories. We also introduce the Robust-Seg dataset, a collection of 688K image-mask pairs with different degradations designed to train and evaluate our model optimally. Extensive experiments across various segmentation tasks and datasets confirm RobustSAM's superior performance, especially under zero-shot conditions, underscoring its potential for extensive real-world application. Additionally, our method has been shown to effectively improve the performance of SAM-based downstream tasks such as single image dehazing and deblurring.
26
+
27
+ <img width="1096" alt="image" src="https://github.com/robustsam/RobustSAM/blob/main/figures/architecture.jpg">
28
+
29
+
30
+ ## Comparison of computational requirements
31
+ <img width="720" alt="image" src='figures/Computational requirements.PNG'>
32
+
33
+ ## Visual Comparison
34
+ <table>
35
+ <tr>
36
+ <td>
37
+ <img src="figures/gif_output/blur_back_n_forth.gif" width="380">
38
+ </td>
39
+ <td>
40
+ <img src="figures/gif_output/haze_back_n_forth.gif" width="380">
41
+ </td>
42
+ </tr>
43
+ <tr>
44
+ <td>
45
+ <img src="figures/gif_output/lowlight_back_n_forth.gif" width="380">
46
+ </td>
47
+ <td>
48
+ <img src="figures/gif_output/rain_back_n_forth.gif" width="380">
49
+ </td>
50
+ </tr>
51
+ </table>
52
+
53
+ <img width="1096" alt="image" src='figures/qualitative_result.PNG'>
54
+
55
+ ## Quantitative Comparison
56
+ ### Seen dataset with synthetic degradation
57
+ <img width="720" alt="image" src='figures/seen_dataset_with_synthetic_degradation.PNG'>
58
+
59
+ ### Unseen dataset with synthetic degradation
60
+ <img width="720" alt="image" src='figures/unseen_dataset_with_synthetic_degradation.PNG'>
61
+
62
+ ### Unseen dataset with real degradation
63
+ <img width="600" alt="image" src='figures/unseen_dataset_with_real_degradation.PNG'>
64
+
65
+ ## Reference
66
+ If you find this work useful, please consider citing us!
67
+ ```python
68
+ @inproceedings{chen2024robustsam,
69
+ title={RobustSAM: Segment Anything Robustly on Degraded Images},
70
+ author={Chen, Wei-Ting and Vong, Yu-Jiet and Kuo, Sy-Yen and Ma, Sizhou and Wang, Jian},
71
+ journal={CVPR},
72
+ year={2024}
73
+ }
74
+ ```
75
+
76
+
77
+ ## Acknowledgements
78
+ We thank the authors of [SAM](https://github.com/facebookresearch/segment-anything) from which our repo is based off of.
79
+