One commonly used loss for imbalanced datasets is inverse frequency cross entropy loss [1]}, [2]} which assigns more weight to the loss of minority classes. Let \(N\)  denote the total number of pixels in the training set and \(N_c\)  denotes the number of pixels belonging to class \(c\in \lbrace 1,..,C\rbrace \) . The frequency of a class is calculated as \(freq(c) = N_c/N\) . We show that while the unweighted cross entropy loss optimizes the overall confidence, the loss weighted by inverse frequency optimizes mean confidence.
If we use an inverse frequency weighting, the loss is rebalanced. Note we leave out the \(N\)  in \(freq(c)\)  as it is shared by all classes.
\(InvCE &= -\sum _{c=1}^C \frac{1}{freq(c)} N_c \log (P^c) = -\sum _{c=1}^C \frac{1}{N_c} N_c \log (P^c) = -\sum _{c=1}^C \log (P^c)\) 
