Mask Generation
sam2
nielsr HF staff commited on
Commit
bae18b4
1 Parent(s): fce35c2

Add code snippets

Browse files

This PR adds code snippets.

Files changed (1) hide show
  1. README.md +34 -5
README.md CHANGED
@@ -1,20 +1,49 @@
1
  ---
2
  license: apache-2.0
 
3
  library_name: sam2
4
  ---
5
 
6
-
7
  Repository for SAM 2: Segment Anything in Images and Videos, a foundation model towards solving promptable visual segmentation in images and videos from FAIR. See the [SAM 2 paper](https://arxiv.org/abs/2408.00714) for more information.
8
 
9
-
10
  The official code is publicly release in this [repo](https://github.com/facebookresearch/segment-anything-2/).
11
- To download the SAM 2 (Hiera-L) checkpoint:
12
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  ```
14
- from huggingface_hub import hf_hub_download
15
- hf_hub_download(repo_id = "facebook/sam2-hiera-large", filename="sam2_hiera_large.pt", local_dir = "./")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  ```
17
 
 
 
18
  ### Citation
19
 
20
  To cite the paper, model, or software, please use the below:
 
1
  ---
2
  license: apache-2.0
3
+ pipeline_tag: mask-generation
4
  library_name: sam2
5
  ---
6
 
 
7
  Repository for SAM 2: Segment Anything in Images and Videos, a foundation model towards solving promptable visual segmentation in images and videos from FAIR. See the [SAM 2 paper](https://arxiv.org/abs/2408.00714) for more information.
8
 
 
9
  The official code is publicly release in this [repo](https://github.com/facebookresearch/segment-anything-2/).
 
10
 
11
+ ## Usage
12
+
13
+ For image prediction:
14
+
15
+ ```python
16
+ import torch
17
+ from sam2.sam2_image_predictor import SAM2ImagePredictor
18
+
19
+ predictor = SAM2ImagePredictor.from_pretrained("facebook/sam2-hiera-large")
20
+
21
+ with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
22
+ predictor.set_image(<your_image>)
23
+ masks, _, _ = predictor.predict(<input_prompts>)
24
  ```
25
+
26
+ For video prediction:
27
+
28
+ ```python
29
+ import torch
30
+ from sam2.sam2_video_predictor import SAM2VideoPredictor
31
+
32
+ predictor = SAM2VideoPredictor.from_pretrained("facebook/sam2-hiera-large")
33
+
34
+ with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
35
+ state = predictor.init_state(<your_video>)
36
+
37
+ # add new prompts and instantly get the output on the same frame
38
+ frame_idx, object_ids, masks = predictor.add_new_points_or_box(state, <your_prompts>):
39
+
40
+ # propagate the prompts to get masklets throughout the video
41
+ for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
42
+ ...
43
  ```
44
 
45
+ Refer to the [demo notebooks](https://github.com/facebookresearch/segment-anything-2/tree/main/notebooks) for details.
46
+
47
  ### Citation
48
 
49
  To cite the paper, model, or software, please use the below: