Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification
It is generally believed that the human visual system is biased towards the
recognition of shapes rather than textures. This assumption has led to a
growing body of work aiming to align deep models' decision-making processes
with the fundamental properties of human vision. The reliance on shape features
is primarily expected to improve the robustness of these models under covariate
shift. In this paper, we revisit the significance of shape-biases for the
classification of skin lesion images. Our analysis shows that different skin
lesion datasets exhibit varying biases towards individual image features.
Interestingly, despite deep feature extractors being inclined towards learning
entangled features for skin lesion classification, individual features can
still be decoded from this entangled representation. This indicates that these
features are still represented in the learnt embedding spaces of the models,
but not used for classification. In addition, the spectral analysis of
different datasets shows that in contrast to common visual recognition,
dermoscopic skin lesion classification, by nature, is reliant on complex
feature combinations beyond shape-bias. As a natural consequence, shifting away
from the prevalent desire of shape-biasing models can even improve skin lesion
classifiers in some cases.