
EndoVis2017 ([1]}): dataset from the 2017 MICCAI EndoVis Robotic Instrument Segmentation Challenge. The dataset contains 10 video clips of abdominal porcine procedures, performed using da Vinci Xi systems. Each video contains a total of 300 high-resolution frames (1280 × 1024), recorded at 2 Hz. In the challenge 8x 225 frames were used for training, while the remaining 8x 75 frames and another 2x 300 frames were held out by the organizers for testing. According to the challenge rules, man-made devices not belonging to the da Vinci system (e.g. drop-in Ultra-Sound probe), labelled by the organizers as part of a class called Other, are to be included in the background class for the binary segmentation task. This introduces the need for a model to perform a semantic differentiation inside the instrument class (da Vinci instruments and Other instruments), which goes beyond the scope of motion-based segmenters. For this reason we refer to the dataset labelled according to the challenge rules as EndoVis2017Challenge, and also consider a second version of it, called EndoVis2017VOS, where both da Vinci and other man-made devices are labelled as instrument. For the main experiments, we report results on both.
We provide results on this dataset according to 2 modalities: 1) following the same evaluation protocol as [2]}, by performing 4-fold cross-validation on the 8x 225 released training data (regrouped in 4 splits), and reporting the average metric on the 4 splits, for direct and fair comparison with other state-of-the-art approaches; 2) by training on RandSurg, a dataset of unlabelled data, described below, and testing on the 8x 225 EndoVis2017VOS frames.

RandSurg: this dataset consists of 4 full unlabelled laparoscopic robotic-assisted procedures downloaded from a public repository ([3]}): adhesiolysis (1036 frames), inguinal hernia repair (1075 frames), appendectomy (500 frames) and ex-vivo suturing demo (525 frames). A set of experiments was carried out by training our model on this dataset and evaluating the performance on EndoVis2017VOS; in order to simulate a realistic application of the FUN-SIS method, and show its ease-of-use, the videos underwent minimal pre-processing (cropping, no trimming, so possibly including out-of-body scenes).

STRAS: this dataset is obtained from endoscopic submucosal dissection procedures performed through the STRAS robotic system ([4]}), a robotic system consisting of a robotized endoscope, having two lateral channels for flexible robotic tools. The dataset was built from a 5 days-experiment on porcine modelsThe study protocol for this experiment was approved by the Institutional Ethical Committee on Animal Experimentation (ICOMETH No.38.2011.01.018). Animals were managed in accordance with French laws for animal use and care as well as with the European Community Council directive no. 2010/63/EU ([5]}), recorded at 30 fps. Each frame was paired with another 1 second apart in the future, for optical-flow computation. The whole dataset was resampled regularly, yielding a total of 5644 frames (\(\sim \) 1100 per experiment day). For each day, 200 frames, regularly spaced, were manually annotated for evaluation (1000 annotated samples in total). The dataset contains challenging sequences, involving bleeding, smoke, strong tool-tissue interaction and image blurring. We provide results on this dataset by performing 5-fold cross-validation (each fold corresponding to an experiment day), and reporting the average metric on the 5 splits.

Cholec80 ([6]}): dataset containing 80 unlabelled videos of manual laparoscopic cholecystectomy procedures captured at 25 Hz and resampled at 1 Hz. We provide qualitative results on this dataset by using the standard split (40 videos for training, 40 videos for testing) to show cross-surgery applicability of the proposed FUN-SIS method.

DAVIS2016 ([7]}): a popular VOS dataset, containing different moving objects (e.g. animals, people, cars). The dataset consists of 50 clips for a total of 3455 1080p frames with pixel-wise annotations. We provide results on this dataset in order to evaluate the proposed optical-flow segmentation approach on non-surgical videos. To this aim, the standard training-test split was used (30 videos for training and 20 for testing), for fair comparison with state-of-the-art VOS approaches.

