In this paper, we propose a novel deep learning architecture for joint classification and localization of abnormalities in mammograms. We first assume a weakly supervised setting and present a new approach with data driven decisions. This novel network combines two learning branches with region-level classification and region ranking. The network provides a global classification of the image into multiple classes, such as malignant, benign or normal. Our method further enables the localization of abnormalities as global class discriminative regions in full mammogram resolution. Next, we extend this method to a semi-supervised setting that engages a small set of local annotations, using a novel architecture, and a multi-task objective function. We present the impact of the local annotations on several performance measures, including localization, to evaluate the cost effectiveness of lesion annotation effort. Our evaluation is made over a large multi-center mammography dataset of @math 3,000 mammograms with various findings. Experimental results demonstrate the capabilities and advantages of the proposed method over previous weakly-supervised strategies, and the impact of semi-supervised learning. We show that targeting the annotation of only 5 of the images can significantly boost performance.
These methods deal with fusion of weak labels and a subset of data with local annotations, namely fully labeled (also known as strongly labeled) data. @cite_30 recently proposed a method for training Fast RCNN @cite_35 via Expectation-Maximization (EM). Focusing on the detection problem, they treated instance-level labels as missing data for weakly annotated images. Their method alternated between two steps: 1) E-step: estimating a probability distribution over all possible latent locations, and 2) M-step: updating a CNN using estimated locations from the last E-step. In the M-step, they optimize the sum of a region-level likelihood function from the fully supervised images and the estimation from the E-step. Their method was applied on non-medical (natural) images, and in practice, the quality of the final solution depended heavily on the initialization by another method ( @cite_24 , which we compare our method with). Furthermore, their approach required thousands of Fast RCNN training iterations at each M-step, that is computationally expensive, particularly for large images such as mammograms.
@cite_30 @cite_35 @cite_24
2592384071 2950489286 2617840581 2949561789
Object detection when provided image-level labels instead of instance-level labels (i.e., bounding boxes) during training is an important problem in computer vision, since large scale image datasets with instance-level labels are extremely costly to obtain. In this paper, we address this challenging problem by developing an Expectation-Maximization (EM) based object detection method using deep convolutional neural networks (CNNs). Our method is applicable to both the weakly-supervised and semi-supervised settings. Extensive experiments on PASCAL VOC 2007 benchmark show that (1) in the weakly supervised setting, our method provides significant detection performance improvement over current state-of-the-art methods, (2) having access to a small number of strongly (instance-level) annotated images, our method can almost match the performace of the fully supervised Fast RCNN. We share our source code at this https URL Despite the recent advances in automatically describing image contents, their applications have been mostly limited to image caption datasets containing natural images (e.g., Flickr 30k, MSCOCO). In this paper, we present a deep learning model to efficiently detect a disease from an image and annotate its contexts (e.g., location, severity and the affected organs). We employ a publicly available radiology dataset of chest x-rays and their reports, and use its image annotations to mine disease names to train convolutional neural networks (CNNs). In doing so, we adopt various regularization techniques to circumvent the large normal-vs-diseased cases bias. Recurrent neural networks (RNNs) are then trained to describe the contexts of a detected disease, based on the deep CNN features. Moreover, we introduce a novel approach to use the weights of the already trained pair of CNN RNN on the domain-specific image text dataset, to infer the joint image text contexts for composite image labeling. Significantly improved image annotation results are demonstrated using the recurrent neural cascade model by taking the joint image text contexts into account. Conventional change detection methods require a large number of images to learn background models or depend on tedious pixel-level labeling by humans. In this paper, we present a weakly supervised approach that needs only image-level labels to simultaneously detect and localize changes in a pair of images. To this end, we employ a deep neural network with DAG topology to learn patterns of change from image-level labeled training data. On top of the initial CNN activations, we define a CRF model to incorporate the local differences and context with the dense connections between individual pixels. We apply a constrained mean-field algorithm to estimate the pixel-level labels, and use the estimated labels to update the parameters of the CNN in an iterative EM framework. This enables imposing global constraints on the observed foreground probability mass function. Our evaluations on four benchmark datasets demonstrate superior detection and localization performance. We consider the task of learning a classifier for semantic segmentation using weak supervision in the form of image labels which specify the object classes present in the image. Our method uses deep convolutional neural networks (CNNs) and adopts an Expectation-Maximization (EM) based approach. We focus on the following three aspects of EM: (i) initialization; (ii) latent posterior estimation (E-step) and (iii) the parameter update (M-step). We show that saliency and attention maps, our bottom-up and top-down cues respectively, of simple images provide very good cues to learn an initialization for the EM-based algorithm. Intuitively, we show that before trying to learn to segment complex images, it is much easier and highly effective to first learn to segment a set of simple images and then move towards the complex ones. Next, in order to update the parameters, we propose minimizing the combination of the standard softmax loss and the KL divergence between the true latent posterior and the likelihood given by the CNN. We argue that this combination is more robust to wrong predictions made by the expectation step of the EM method. We support this argument with empirical and visual results. Extensive experiments and discussions show that: (i) our method is very simple and intuitive; (ii) requires only image-level labels; and (iii) consistently outperforms other weakly-supervised state-of-the-art methods with a very high margin on the PASCAL VOC 2012 dataset.