We utilized the Grad-CAM localization mapping method [1]} to provide a deep insight into the intermediate layers and identify what components in a CT image have obtained the most attention by the model. The Grad-CAM method extracts the spatial information, which is preserved by the convolutional layers and specifies the parts in the image having the most contribution to the final prediction. More specifically, the Grad-CAM method generates a localization heatmap corresponding to each layer and the target class to determine the locations to which the model paid the most attention. This localization heatmap is derived by a weighted average of all feature maps in the convolutional layer followed by a Rectified Linear Unit (ReLU) activation function.
