Segmentation is an essential component in many image analysis pipelines that aim to extract clinically useful information from medical images to inform clinical decisions in diagnosis, treatment planning, or monitoring of disease progression. A multitude of approaches have been proposed for solving segmentation problems, with popular techniques based on graph cuts [1]}, multi-atlas label propagation [2]}, statistical models [3]}, and supervised classification [4]}. Traditionally, performance of a segmentation method is evaluated on an annotated database using various evaluation metrics in a cross-validation setting. These metrics reflect the performance in terms of agreement [5]} of a predicted segmentation compared to a reference `ground truth' (GT)For simplicity, we use the term ground truth to refer to the best known reference, which is typically a manual expert segmentation.. Commonly used metrics include Dice's similarity coefficient (DSC) [6]} and other overlap based measures [7]}, but also metrics based on volume differences, surface distances, and others [8]}, [9]}, [10]}. A detailed analysis of common metrics and their suitability for segmentation evaluation can be found in [11]}.
