Analyzing the Effects of Handling Data Imbalance on Learned Features from Medical Images by Looking Into the Models

One challenging property lurking in medical datasets is the imbalanced data distribution, where the frequency of the samples between the different classes is not balanced. Training a model on an imbalanced dataset can introduce unique challenges to the learning problem where a model is biased towards the highly frequent class. Many methods are proposed to tackle the distributional differences and the imbalanced problem. However, the impact of these approaches on the learned features is not well studied. In this paper, we look deeper into the internal units of neural networks to observe how handling data imbalance affects the learned features. We study several popular cost-sensitive approaches for handling data imbalance and analyze the feature maps of the convolutional neural networks from multiple perspectives: analyzing the alignment of salient features with pathologies and analyzing the pathology-related concepts encoded by the networks. Our study reveals differences and insights regarding the trained models that are not reflected by quantitative metrics such as AUROC and AP and show up only by looking at the models through a lens.


Introduction
Medical imaging datasets often appear in imbalanced distribution where the frequency of the samples between the different classes of the training dataset is not similar or balanced. The low amount of training samples for infrequent classes or tailed distribution makes it challenging to learn optimal classification boundaries in the representation and can lead to a biased model. Existing methods to tackle class imbalance problem either modify data distribution or learn denotes equal contribution 1 https://github.com/CAMP-eXplain-AI/imba-explain appropriate costs to re-weight class errors. At the data level, the objective is to balance the data distribution through re-sampling techniques which often are prone to over-fitting [18,3,6]. On the other hand, the cost-sensitive approaches modify the learning algorithm to alleviate the bias towards frequent classes or head of distribution [5,26]. The efficacy of these approaches is demonstrated by conventional metrics such as precision and recall and their derivatives. However, the effect of these approaches on the learned features is not well studied. The study of learned features not only improves our understanding of what happens within the models, is specifically insightful when the conventional evaluation metrics do not reflect the effect of applying such cost-sensitive approaches.
This study explores what happens to the learned features when it is trained by cost-sensitive approaches to handle the data imbalance. To understand the effect on learned features, we analyze the internal units of neural networks. Specifically, we analyze the feature maps (outputs of convolutional layers) in deep layers by class activation maps and network dissection [1]. First, we optimize models (ResNet [7] and DenseNet [9]) with Binary Cross-Entropy (BCE) [23]. Several recent and popular cost-sensitive losses such as Weighted BCE (WBCE) [25], Focal loss [21], and Class-Balanced Focal loss (CB-Focal) [5] on NIH Chest Xray [31] dataset and report classical metrics: area Under ROC curve (AUROC), average precision (AP) and predicted probabilities. Then, we visually analyze the impact on salient learned features using class activation maps [33,28] and provide quantitative evaluations to validate the visual observation. We then proceed to analyze the learned features using network dissection [1,16] which quantitatively identifies the concepts encoded (learned) by the model. Statement of Contribution By placing models trained with different learning strategies under the lens, we observe that while metrics such as AUROC and AP report equivalent performance, the models trained with cost-sensitive losses encode more pathology-related concepts. Moreover, we observe an increased alignment between salient learned features and pathology-related features.

Related Works
Handling Data Imbalance: Much of the recent works on imbalanced learning focused on alleviating this problem using novel objective function. Lin et. al. [21] introduce a Focal loss for dense object detection where class-specific weights were automatically learned. Cui et. al. [5] re-weight the loss by the inverse effective number of examples to learn balanced representations. Similarly, [18] modified the weights according the uncertainty of predictions. Others address this problem by multi-task learning [10,26] that used selective instances for training on imbalanced sets for each task. Interpreting Neural Networks: Two principal neural network interpretation approaches are feature attribution [29,32,13,30,22,17,12] (i.e. saliency methods [4]) and analyzing internal units (e.g. feature visualization [24] and dissection [2,16]). Here, we look into the models from both perspectives. We are interested in both the contribution of input features to the output (i.e., feature attribution) and concepts encoded by the network (via analyzing units).
The are many attribution methods, however the identified important features are different for different methods [20,32,15,14]. This is a caveat for researchers using the attribution toolkit. Within the various feature attribution methods, CAM [28] being a classic and intuitive method, it satisfies benchmarks in terms of faithfulness [8,32,15]. Moreover, regardless of its advantages for attribution, the method provides a summary of activation maps in a certain layer by performing a weighted sum of the attributions. We use network dissection [1,16] to identify concepts associated with individual convolutional feature maps.

Method
The question we explore in this paper is what happens to the learned features within the model when we apply cost-sensitive approaches to handle the class imbalance during training. Specifically, we study the intra-class data imbalance, i.e. the imbalance between the number of positive and negative samples for each class. We first introduce the cost-sensitive methods (see Section 3.1) that consider the data imbalance between positive and negative samples. Then, we explain our approaches for analyzing the learned features (see Section 3.2).

Handling Data Imbalance
, our goal is to learn the optimal boundary θ * obtained by empirical loss minimization L D (θ) on the training set D as: θ * = arg min θ L D (θ). The class imbalanced problem exists when the frequency of the samples among different categories are extremely mismatched. Therefore θ learned on D using conventional empirical loss can be biased towards the low frequent classes and significantly different from the ideal boundary θ * . In other words, because of the imbalanced proportion between classes, the optimal boundary is more likely to afford a higher empirical error than an alternative hypothesis based on an empirical loss. This is due to the nature of imbalanced class distribution that forces the classifier to shift θ closer to low-frequent classes because it reduces empirical error. A principal strategy to handle data imbalance is through the cost-sensitive loss functions [21,5,10,27].
Here, we study data imbalance problem in multi-label and binary classifica- m of class m for sample x (i) given by a neural network is defined with multiple outputs as cross entropy error between predicted value and true label using Binary Cross Entropy (BCE) loss. The cost-sensitive losses applied to class can be summarized into a unified framework: where w (i) − are the positive and negative class weights for sample x (i) , respectively. We omit the class index m for simplicity. The four losses considered in this work can be derived as follows: where N + , N − are the number of positive and negative samples of class m, and α, γ and β are hyper-parameters. As we are discussing intra-class imbalance, each loss is considered individually and we apply the cost-sensitive methods on each loss to strike a balance between positive and negative samples for each class.

Looking into the Models
The objective is to analyze the features learned by the model in different training scenarios. In this work, we analyze the internal units of the model, specifically the activation pattern of convolutional feature maps in deeper layers. The deep convolutional layers are chosen due to the following two reasons: 1) Final layers in principle encode higher-level concepts and we are interested to know if these concepts align with pathological features. 2) We focus on convolutional layers instead of deeper fully connected layers, since convolutional layers keep the spatial correspondence with input features, and therefore, we can leverage image annotations from experts to study if the activations align with pathology related features. We analyze the internal features maps from two different perspectives: Class Activation Maps [33,28] can be deemed as a tool that summarizes the contributions of feature maps of the final convolutional layer in one tensor (for the networks under investigation, GradCAM [28], and CAM [33] are equivalent). The method combines the feature maps using their associated weight (or gradient of output with respect to feature map in GradCAM) of their connections to the output, thus summarizing how feature maps contribute to the final output. The method, in essence, shows which areas are contributing to the output for each sample input. Thus using annotations from experts, we can check whether the salient regions are aligned with the pathologies [31,11].
Analyzing Encoded Concepts [1,16]: We use an approach inspired by network dissection [1] which is a tool for individually analyzing the convolutional feature maps of the neural network. In essence, it identifies the concepts associated with each feature map, thus allowing for understanding what concepts are encoded by the network during training. In this work, we use a conceptually similar approach. For a chosen convolutional layer, let A c denote the activation (feature) map of channel c. Similar to [1], we first apply a threshold τ c with P(A C ≥ τ c ) = q to activation values of each channel c to filter the dataset-wise significant activation values, where q is a pre-defined hyper-parameter. For each image in the annotation dataset we compute the corresponding A c and apply the threshold τ c . Subsequently, we identify the connected components in the threshold activations. If the connected component overlaps the bounding box region, we consider the connected component as a pathology related concept. Using this approach we can report two values that reflect the concepts encoded in the network quantitatively, which are explained in Sec. 4.

Experiments
Experimental Setup In this paper, we examine and perform two different datasets and learning tasks: multi-label classification and binary classification on the NIH Chest X-ray dataset [31]. For binary classification. We define samples with the "No Finding" label as "Healthy" samples, and samples with other labels as "Unhealthy" samples. Random crop, horizontal flip, and color jitter are used as the augmentations. In the end, the augmented images are resized to 224 × 224 before being fed into an image classifier.
We use two ImageNet pre-trained classifiers: ResNet50 [7] and DenseNet [9]. The last feature maps of ResNet50 have the size 7 × 7. In order to have a finegrained analysis on smaller features such as Nodule (for which 7 × 7 is a coarse resolution) we use a truncated version of DenseNet. We discard the last two dense blocks and the associated transition blocks, so that the last feature maps have size 28 × 28. We refer to this truncated variant of DenseNet as T-DenseNet. We adopt the same training configurations for all models. Specifically, we train the classifiers on 4 GPUs (DGX-A100) for 50 epochs using the Adam [19] optimizer with weight decay 10 −6 and initial learning rate 0.0004. The learning rate is decayed following the cosine-annealing policy. In addition, we set the batch size to 512. For the Focal loss, the γ and α set to 2.0 and 0.25, respectively; for the class-balanced focal loss, we set γ to 2.0 and β to 0.9999.

Feature Alignment Analysis
The objective is to have a metric for measuring the alignment between the contributing features (from CAMs) for a single prediction and the pathologies. This can be achieved by classical object detection metrics IoBB(Intersection over Bounding Box) and IoR(Intersection of detected Region). IoBB and IoR can be considered as the visual counterparts of recall and precision. Computing these metrics requires thresholding the CAMs. In order to reduce the sensitivity of the results to the chosen threshold, we implement soft IoBB and IoR computation. This is performed by normalizing the CAMs to [0, 1] and applying a summation of values within the regions under consideration (intersection, bounding box, detected region). for all images and normalize by the number of images. The provided metrics reflect how individual feature maps are aligned with pathologies.

Results and Discussion
Performance Analysis We evaluate the classification performance using AU-ROC and AP (Fig. 2). The performance differences between BCE and weighted losses (WBCE, Focal, and CB-Focal) are marginal. Therefore the effect of handling data imbalance via losses is not visible via these metrics. We also report the predicted probabilities of different losses. The predicted probabilities of the weighted losses are substantially higher than that of the BCE. This is expected as more weight is given to positive samples. Moreover, as more weight is given to the positive weights, the recall is higher. However, it is not clear if the increased recall is due to model's inclination towards identifying samples as positive or due to the model having learned and leveraging more predictive features.

Feature Alignment Analysis
In this section, we analyze the models' internal feature maps to see if the increased recall and probability values are related to predictive features relevant to the pathologies. This is realized by comparing CAMs and the salient features for individual predictions with annotations from radiologists. In Fig. 1 we observe that in models trained with cost-sensitive losses, the additionally activated regions are aligned with the pathologies (bounding boxes). This shows that the increased recall is not due to the models' propensity towards identifying samples as positive but due to having learned meaningful Task  Multi-label  Binary  Classifier T-DenseNet  ResNet50  T-DenseNet  ResNet50  Metric IoBB  IoR  IoBB  IoR  IoBB  IoR  IoBB  IoR  BCE Table 1: Feature Alignment Analysis. The average IoBB and IoR results for test sets images that have bounding box annotations. These metrics are provided to quantitatively validate the observation in Fig. 1   predictive features. Note that metrics such as AUROC and AP are roughly equivalent for all models (Fig. 2). In order to quantify the observed behavior in Fig. 1 we report IoBB and IoR results in Tab. 1. The increased IoBB for models trained with cost-sensitive losses shows that the features cover more areas of bounding boxes (pathologies). We also observe a decrease in IoR, which is also evident in Fig. 1 as falsely positive active regions have also increased.

Analysis of Concepts
This section analyzes the effect of handling data imbalance on the concepts encoded by the model. This is performed by counting the number of detected pathology-related concepts. In Tab. 2 we observe that the models trained with cost-sensitive losses consistently have more detectors. Thus it can be concluded that the models encode more pathology-related concepts when the data imbalance is considered in the loss.

Conclusion
In this work, we study the effect of handling data imbalance using cost-sensitive losses during training on the learned feature maps. The feature maps are analyzed from two perspectives: class activation maps and the concept encoded by each feature map. We observe that although classical metrics such as AUROC and AP report equivalent performance for the studied models, the learned features are different in these models. When data imbalance is handled, the models encode more pathology-related concepts. Moreover, we observe that overall increased recall and predicted probability of these models are accompanied by an increased alignment between learned features and pathology-related features.