Next, complementary to the findings in [1]}, we observe that models based on the same architecture (e.g. DenseNet121) learn quite similar distributions when compared to each other but differ substantially when compared to other architectures: all CompNets learn similar coefficient distributions although trained on different reconstruction planes; Albeit trained in different regimes, all LungMask models also form similar distributions to the standalone UNet. Still, some shifts in the distributions can be seen, in particular in the DenseNet121 models from TorchXRayVision which were trained on different datasets. The model trained on the Kaggle RSNA Pneumonia Challenge dataset seems to overrepresent specific coefficients (and therefore filters clusters) by a large amount, while the models trained on other datasets such as MIMIC-CXR [2]}, NIH [3]}, [4]}, and CheXpert [5]} tend to learn coefficient distributions that are smoother and more similar to the distribution of the model that was trained on all datasets. The most salient coefficient distribution is seen in the ResNet50 which was trained on \(512\times 512\)  px images instead of \(256\times 256\)  px. Spikes dominate the KDEs at 0 for all axes, which indicates a highly sparse model. Indeed, by applying a structured pruning on the kernels, we find that only approx. 1% of filters (N=12900) is non-sparse. Due to a lack of access to the datasets, we cannot verify the integrity of this model. However, if this model's performance is on-par with the other models, this opens the question of whether a) smaller and therefore easier-to-train models would not be more suitable for this problem and perhaps brain segmentation in general and b) higher resolution images may be more suitable as they may contain more salient features for classification than their down-sampled counter-parts.
<FIGURE>