Researchers have studied the imbalance problem for classification, detection, and segmentation extensively. Most prior research has been on designing balanced loss functions. We classify existing loss functions under three categories: region-based losses, statistics-balanced losses and performance-balanced losses. Region-based losses directly optimize region metrics (e.g., Jaccard index [1]}) and are mainly popular in medical segmentation applications; Statistics-balanced losses (e.g., LDAM [2]}, Class-Balanced (CB) loss [3]}) up/down weights the contribution of a class based on its class margin or class size; however, they tend to encourage excessive false positives in minority classes to improve mean accuracy especially in segmentation. A recent study in [4]} also shows that the weighting undermines the generic representation learning capability of the feature extractors; Performance-balanced losses (e.g., focal loss [5]}) use a certain performance indicator to weight the loss of each class. As an example, focal loss assumes that the difficulty of a class is correlated with imbalance and can be reflected by the predicted confidence. However, it has not been very successful in other applications for dealing with imbalance as reported by [3]}. We investigate the reasons of failure in Sec. REF .
