Convolutional neural networks (CNNs) deliver exceptional results for computer vision, including medical image analysis. With the growing number of available architectures, picking one over another is far from obvious. Existing art suggests that, when performing transfer learning, the performance of CNN architectures on ImageNet correlates strongly with their performance on target tasks. We evaluate that claim for melanoma classification, over 9 CNNs architectures, in 5 sets of splits created on the ISIC Challenge 2017 dataset, and 3 repeated measures, resulting in 135 models. The correlations we found were, to begin with, much smaller than those reported by existing art, and disappeared altogether when we considered only the top-performing networks: uncontrolled nuisances (i.e., splits and randomness) overcome any of the analyzed factors. Whenever possible, the best approach for melanoma classification is still to create ensembles of multiple models. We compared two choices for selecting which models to ensemble: picking them at random (among a pool of high-quality ones) vs. using the validation set to determine which ones to pick first. For small ensembles, we found a slight advantage on the second approach but found that random choice was also competitive. Although our aim in this paper was not to maximize performance, we easily reached AUCs comparable to the first place on the ISIC Challenge 2017.
The best architectures for skin lesion classification remain, thus, an open issue. The Challenge results do not allow analyzing a single factor from a multitude of choices among participants. The study of Valle al @cite_15 , although exhaustive for 9 factors, evaluates only two levels for each factor, picking ResNet-101 and Inception-v4 for architectures. No other study, as far as we know, attempts to answer the question systematically.
@cite_15
2778722845 2602631755 2789357239 2604384166
Deep learning methods for image analysis have shown impressive performance in recent years. In this paper, we present deep learning based approaches to solve two problems in skin lesion analysis using a dermoscopic image containing skin tumor. In the first problem, we use a fully convolutional-deconvolutional architecture to automatically segment skin tumor from the surrounding skin. In the second problem, we use a simple convolutional neural network and VGG-16 architecture using transfer learning to address the two different tasks in skin tumor classification. The proposed models are trained and evaluated on standard benchmark datasets from the International Skin Imaging Collaboration (ISIC) 2017 Challenge, which consists of 2000 training samples and 600 testing samples. The result shows that the proposed methods achieve promising performances. In the first problem, the average value of Jaccard index for lesion segmentation using fully convolutional-deconvolutional architecture is 0.507. In the second problem, the values of area under the receiver operating characteristic curve (AUC) on two different lesion classifications using VGG16 with transfer learning are 0.763 and 0.869, respectively; the average value of AUC in two tasks is 0.816. This extended abstract describes the participation of RECOD Titans in parts 1 and 3 of the ISIC Challenge 2017 "Skin Lesion Analysis Towards Melanoma Detection" (ISBI 2017). Although our team has a long experience with melanoma classification, the ISIC Challenge 2017 was the very first time we worked on skin-lesion segmentation. For part 1 (segmentation), our final submission used four of our models: two trained with all 2000 samples, without a validation split, for 250 and for 500 epochs respectively; and other two trained and validated with two different 1600 400 splits, for 220 epochs. Those four models, individually, achieved between 0.780 and 0.783 official validation scores. Our final submission averaged the output of those four models achieved a score of 0.793. For part 3 (classification), the submitted test run as well as our last official validation run were the result from a meta-model that assembled seven base deep-learning models: three based on Inception-V4 trained on our largest dataset; three based on Inception trained on our smallest dataset; and one based on ResNet-101 trained on our smaller dataset. The results of those component models were stacked in a meta-learning layer based on an SVM trained on the validation set of our largest dataset. Deep CNN techniques have dramatically become the state of the art in image classification. However, applying high-capacity Deep CNN in medical image analysis has been impeded because of scarcity of labeled data. This study has two primary contributions: first, we propose a classification model to improve performance of classification of skin lesion using Deep CNN and Data Augmentation. Second, we demonstrate the use of image data augmentation for overcoming the problem of data limitation and examine the influence of different number of augmented samples on the performance of different classifiers. The proposed classification system is evaluated using the largest public skin lesion testing dataset, containing 600 testing images, and 6,162 training images. New state-of-the-art performance result is archived with AUC (89.2 vs. 87.4 ), AP (73.9 vs. 71.5 ), and ACC (89.0 vs. 87.2 ). In additional, we explore the influence of each image augmentation on the three classifiers and observe that performance of each classifier is influenced differently by each augmentation and has better results comparing with traditional methods. Thus, it is suggested that the performance of skin cancer classification and medial image classification could be improved further by applying data augmentation. This short paper reports the method and the evaluation results of Casio and Shinshu University joint team for the ISBI Challenge 2017 - Skin Lesion Analysis Towards Melanoma Detection - Part 3: Lesion Classification hosted by ISIC. Our online validation score was 0.958 with melanoma classifier AUC 0.924 and seborrheic keratosis classifier AUC 0.993.