When the training data distribution is long-tailed, e.g., thousands of samples per “head” class vs. just a few samples per “tail” class, the resultant classification model will inevitably downplay the minor “tail” and overplay the major “head”. Therefore, to lift up the “tail” performance, recent methods focus on the trade-off between the “head” and “tail” contributions, e.g., sample re-weighting [1]}, [2]}, balanced training loss [3]}, [4]}, and “head” causal effect removal [5]}. As you may expect, the “tail” rise costs the “head” fall; but fortunately, the overall gain is still positive. The reasons are two-fold. First, thanks to the abundant “head” training samples, the strong “head” features
can afford the trade-off, making the “head” performance drop much less significant than the “tail” rise. Second, the test distribution is often assumed to be balanced, i.e., “head” and “tail” have the same number of samples, so the “head” fall is not amplified by the balanced sample number.
