---
license: mit
base_model:
- ybelkada/segment-anything
pipeline_tag: zero-shot-image-classification
tags:
- medical
---
<!--
 * @Author: Chris Xiao yl.xiao@mail.utoronto.ca
 * @Date: 2023-09-12 22:10:18
 * @LastEditors: Chris Xiao yl.xiao@mail.utoronto.ca
 * @LastEditTime: 2023-12-09 17:15:37
 * @FilePath: /EndoSAM/README.md
 * @Description: 
 * I Love IU
 * Copyright (c) 2023 by Chris Xiao yl.xiao@mail.utoronto.ca, All Rights Reserved. 
-->
# EndoSAM
Fine-tune for endoscope clapster segmentation (adapted from [SurgicalSAM](https://github.com/wenxi-yue/SurgicalSAM) but provided all scripts for fine-tune)
<!-- TOC -->

- [EndoSAM](#endosam)
    - [Installation tested on Ubuntu 20.04.6 LTS x86_64](#installation-tested-on-ubuntu-20046-lts-x86_64)
    - [Usage](#usage)
    - [Inference](#inference)
    - [Reference](#reference)

<!-- /TOC -->
## Installation (tested on Ubuntu 20.04.6 LTS x86_64)
```
git clone https://huggingface.co/ChrisXiao/EndoSAM
cd EndoSAM
conda env create -f environment.yml
conda activate sam
```
If conda cannot install successfully, try
 ```
conda create -y -n sam python=3.10.11
conda activate sam
pip install -r requirements.txt
```

## Usage
- Download (using ```wget``` or manual way) the SAM model checkpoint and place it into ```sam_weights```folder, click the links below to download the checkpoint for the corresponding model type.

    - **`default` or `vit_h`: [ViT-H SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth)**
    - `vit_l`: [ViT-L SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth)
    - `vit_b`: [ViT-B SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth)
- Run the script (change the config file for play)
```
cd endoSAM
python train.py --cfg ../config/finetune.yaml
```
- GPU RAM Requirement\
Even though this is the fine-tune work, it requires a large GPU RAM. We tested on the [EndoVis2017](https://endovissub2017-roboticinstrumentsegmentation.grand-challenge.org/) [1] and [EndoVis2018](https://endovissub2018-roboticscenesegmentation.grand-challenge.org/) [2] Dataset and image resolution is 1024 x 1024 with initial processing for the SAM. **Use suitable batch size based on the VRAM you have**
    - Batch Size 1 -> 6 GB RAM
    - Batch Size 2 -> 12 GB RAM
    - Batch Size 4 -> 21 GB RAM
    - Batch Size 8 -> 33 GB RAM

The ```training checkpoints, best model, loss plots and log files``` will be saved in the```log_folder```, ```model_folder```, ```ckpt_folder``` and ```plot_folder```you provide in the config file respectively.
## Inference
```
python test.py --cfg ../config/finetune.yaml
```
The prediction results will be saved into the ```test_folder``` you provide in the config file.

## Reference
[1] Allan, M.; Shvets, A.; Kurmann, T.; Zhang, Z.; Duggal, R.; Su, Y.-H.; Rieke, N.; Laina, I.; Kalavakonda, N.; Bodenstedt, S.; Herrera, L.; Li, W.; Iglovikov, V.; Luo, H.; Yang, J.; Stoyanov, D.; Maier-Hein, L.; Speidel, S.; and Azizian, M. 2019. 2017 Robotic Instrument Segmentation Challenge. arXiv:1902.06426.

[2] Allan, M.; Kondo, S.; Bodenstedt, S.; Leger, S.; Kadkhodamohammadi, R.; Luengo, I.; Fuentes, F.; Flouty, E.; Mohammed, A.; Pedersen, M.; Kori, A.; Alex, V.; Krishnamurthi, G.; Rauber, D.; Mendel, R.; Palm, C.; Bano, S.; Saibro, G.; Shih, C.-S.; Chiang, H.-A.; Zhuang, J.; Yang, J.; Iglovikov, V.; Dobrenkii, A.; Reddiboina, M.; Reddy, A.; Liu, X.; Gao, C.; Unberath, M.; Kim, M.; Kim, C.; Kim, C.; Kim, H.; Lee, G.; Ullah, I.; Luna, M.; Park, S. H.; Azizian, M.; Stoyanov, D.; Maier-Hein, L.; and Speidel, S. 2020. 2018 Robotic Scene Segmentation Challenge. arXiv:2001.11190.