Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> Summary Ultrasound examination of the breast has recently come to maturity with the availability of high-frequency, hand-held transducers. It is now routinely used in breast imaging centers as an essential complement to physical examination and mammography not only to differentiate cystic from solid masses but also to further characterize solid masses. New diagnostic parameters include the tumor's shape, vascularity, and mobility. Although sonography can detect some carcinomas missed by mammography, its inability to demonstrate microcalcifications and minimal carcinomas and its dependence on operator skill are insurmountable obstacles that prevent its use for cancer screening. New applications of breast ultrasound in cancer management include the refinement of cancer staging through the detection of nonpalpable regional lymph node metastases, the evaluation of response to chemotherapy of locally advanced carcinomas, and the early detection of local recurrences following conservative treatment. Because of its unique real-time capability, sonography is the most accurate and convenient technique for guiding needle biopsy or localization of any nonpalpable breast mass or lymph node visible on sonograms. <s> BIB001 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> This paper describes an approach to boundary detection in ultrasound speckle based on an image enhancement technique. The enhancement algorithm works by filtering the image with "sticks," short line segments which are varied in orientation to achieve the maximum projected value at each point. The statistical properties of this approach have been described in an earlier paper; in this work we present three significant extensions to improve the performance of the basic method. First, we investigate the effect of varying the size and shape of the sticks. We show that these variations affect the performance of the algorithm in very fundamental ways, for example by making it more or less sensitive to thinner or more tightly curving boundaries. Second, we present a means of improving the performance of this technique by estimating the distribution function of the orientation of the line passing through each point. Finally, we show that images can be "stained" for easier visual interpretation by applying to each pixel a false color whose hue is related to the orientation of the most prominent line segment at that point. Examples are given to illustrate the performance of the different settings on a single image. <s> BIB002 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> In this paper we present a computationally efficient segmentation algorithm for breast masses on sonography that is based on maximizing a utility function over partition margins defined through gray-value thresholding of a preprocessed image. The performance of the segmentation algorithm is evaluated on a database of 400 cases in two ways. Of the 400 cases, 124 were complex cysts, 182 were benign solid lesions, and 94 were malignant lesions. In the first evaluation, the computer-delineated margins were compared to manually delineated margins. At an overlap threshold of 0.40, the segmentation algorithm correctly delineated 94% of the lesions. In the second evaluation, the performance of our computer-aided diagnosis method on the computer-delineated margins was compared to the performance of our method on the manually delineated margins. Round robin evaluation yielded Az values of 0.90 and 0.87 on the manually delineated margins and the computer-delineated margins, respectively, in the task of distinguishing between malignant and nonmalignant lesions. <s> BIB003 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> We present a computer-aided diagnosis(CAD) method for breast lesions on ultrasound that is based on the automatic segmentation of lesions and the automatic extraction of four features related to the lesion shape, margin, texture, and posterior acoustic behavior. Using a database of 400 cases (94 malignant lesions, 124 complex cysts, and 182 benign solid lesions), we investigate the marginal benefit of each feature in our CAD method and the performance of our CAD method in distinguishing malignant lesions from various classes of benign lesions. Finally, independent validation is performed on our CAD method. Eleven independent trials yielded an average A z value of 0.87 in the task of distinguishing malignant from benign lesions. <s> BIB004 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> We investigated the use of a radial gradient index (RGI) filtering technique to automatically detect lesions on breast ultrasound. After initial RGI filtering, a sensitivity of 87% at 0.76 false-positive detections per image was obtained on a database of 400 patients (757 images). Next, lesion candidates were segmented from the background by maximizing an average radial gradient (ARD) index for regions grown from the detected points. At an overlap of 0.4 with a radiologist lesion outline, 75% of the lesions were correctly detected. Subsequently, round robin analysis was used to assess the quality of the classification of lesion candidates into actual lesions and false-positives by a Bayesian neural network. The round robin analysis yielded an Az value of 0.84, and an overall performance by case of 94% sensitivity at 0.48 false-positives per image. Use of computerized analysis of breast sonograms may ultimately facilitate the use of sonography in breast cancer screening programs. <s> BIB005 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> To increase the ability of ultrasonographic technology for the differential diagnosis of solid breast tumors, we describe a novel computer-aided diagnosis (CADx) system using neural networks for classification of breast tumors. Tumor regions and surrounding tissues are segmented from the physician-located region-of-interest (ROI) images by applying our proposed segmentation algorithm. Cooperating with the segmentation algorithm, three feasible features, including variance contrast, autocorrelation contrast and distribution distortion of wavelet coefficients, were extracted from the ROI images for further classification. A multilayered perceptron (MLP) neural network trained using error back-propagation algorithm with momentum was then used for the differential diagnosis of breast tumors on sonograms. In the experiment, 242 cases (including benign breast tumors from 161 patients and carcinomas from 82 patients) were sampled with k-fold cross-validation (k = 10) to evaluate the performance. The receiver operating characteristic (ROC) area index for the proposed CADx system is 0.9396 +/- 0.0183, the sensitivity is 98.77%, the specificity is 81.37%, the positive predictive value is 72.73% and the negative predictive value is 99.24%. Experimental results showed that our diagnosis model performed very well for breast tumor diagnosis. <s> BIB006 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> Displayed ultrasound (US) B-mode images often exhibit tissue intensity inhomogeneities dominated by nonuniform beam attenuation within the body. This is a major problem for intensity-based, automatic segmentation of video-intensity images because conventional threshold-based or intensity-statistic-based approaches do not work well in the presence of such image distortions. Time gain compensation (TGC) is typically used in standard US machines in an attempt to overcome this. However this compensation method is position-dependent which means that different tissues in the same TGC time-range (or corresponding depth range) will be, incorrectly, compensated by the same amount. Compensation should really be tissue-type dependent but automating this step is difficult. The main contribution of this paper is to develop a method for simultaneous estimation of video-intensity inhomogeities and segmentation of US image tissue regions. The method uses a combination of the maximum a posteriori (MAP) and Markov random field (MRF) methods to estimate the US image distortion field assuming it follows a multiplicative model while at the same time labeling image regions based on the corrected intensity statistics. The MAP step is used to estimate the intensity model parameters while the MRF step provides a novel way of incorporating the distributions of image tissue classes as a spatial smoothness constraint. We explain how this multiplicative model can be related to the ultrasonic physics of image formation to justify our approach. Experiments are presented on synthetic images and a gelatin phantom to evaluate quantitatively the accuracy of the method. We also discuss qualitatively the application of the method to clinical breast and cardiac US images. Limitations of the method and potential clinical applications are outlined in the conclusion. <s> BIB007 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> Breast cancer is the most frequently diagnosed malignancy and the second leading cause of mortality in women . In the last decade, ultrasound along with digital mammography has come to be regarded as the gold standard for breast cancer diagnosis. Automatically detecting tumors and extracting lesion boundaries in ultrasound images is difficult due to their specular nature and the variance in shape and appearance of sonographic lesions. Past work on automated ultrasonic breast lesion segmentation has not addressed important issues such as shadowing artifacts or dealing with similar tumor like structures in the sonogram. Algorithms that claim to automatically classify ultrasonic breast lesions, rely on manual delineation of the tumor boundaries. In this paper, we present a novel technique to automatically find lesion margins in ultrasound images, by combining intensity and texture with empirical domain specific knowledge along with directional gradient and a deformable shape-based model. The images are first filtered to remove speckle noise and then contrast enhanced to emphasize the tumor regions. For the first time, a mathematical formulation of the empirical rules used by radiologists in detecting ultrasonic breast lesions, popularly known as the "Stavros Criteria" is presented in this paper. We have applied this formulation to automatically determine a seed point within the image. Probabilistic classification of image pixels based on intensity and texture is followed by region growing using the automatically determined seed point to obtain an initial segmentation of the lesion. Boundary points are found on the directional gradient of the image. Outliers are removed by a process of recursive refinement. These boundary points are then supplied as an initial estimate to a deformable model. Incorporating empirical domain specific knowledge along with low and high-level knowledge makes it possible to avoid shadowing artifacts and lowers the chance of confusing similar tumor like structures for the lesion. The system was validated on a database of breast sonograms for 42 patients. The average mean boundary error between manual and automated segmentation was 6.6 pixels and the normalized true positive area overlap was 75.1%. The algorithm was found to be robust to 1) variations in system parameters, 2) number of training samples used, and 3) the position of the seed point within the tumor. Running time for segmenting a single sonogram was 18 s on a 1.8-GHz Pentium machine. <s> BIB008 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> In this study, we made use of the discrete active contour model to overcome the natural properties of ultrasound (US) images, speckle, noise and tissue-related textures, to segment the breast tumors precisely. Determination of the real tumor boundary with the snake-deformation process requires an initial contour estimate. However, the manual way to sketch an initial contour is very time-consuming. Thus, we propose an automatic initial contour-finding method that not only maintains the tumor shape, but also is close to the tumor boundary and inside the tumor. During the deformation process, to prevent the snake trapping into the false position caused by tissue-related texture or speckle, we added the edge information as an image feature to define the external force. In addition, because the 3-D volume of a tumor is essentially constructed by a sequence of 2-D images, our method for finding boundaries of a tumor can be extended to 3-D cases. By precisely counting the volume of the 3-D images, we can get the volume of tumor. Finally, we will show that the proposed techniques have rather good performance and lead to a satisfactory result in comparison with the estimated volume and physician's estimate. <s> BIB009 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> In this paper, we apply the three-dimensional (3-D) active contour model to a 3-D ultrasonic data file for segmenting of the breast tumor. The 3-D ultrasonic file is composed of a series of two-dimensional (2-D) images. Most of traditional techniques of 2-D image segmentation will not use the information between adjacent images. To suit the property of the 3-D data, we introduce the concept of the 3-D stick, the 3-D morphologic process and the 3-D active contour model. The 3-D stick can get over the problem that the ultrasonic image is full of speckle noise and highlight the edge information in images. The 3-D morphologic process helps to determine the contour of the tumor and the resulting contour can be regarded as the initial contour of the active contour model. Finally, the 3-D active contour model will make the initial contour approach to the real contour of the tumor. However, there is emphasis on these 3-D techniques that they do not consist of a series of 2-D techniques. When they work, they will consider the horizontal, vertical and depth directions at the same time. The use of these 3-D techniques not only segments the 3-D shape but also obtains the volume of the tumor. The volume of the tumor calculated by the proposed method will be compared with the volume calculated by the VOCAL software with the physician's manually drawn shape and it shows that the performance of our method is satisfactory. <s> BIB010 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> Real-time needle segmentation and tracking is very important in image-guided surgery, biopsy, and therapy. Due to its robustness to the addition of extraneous noise, the Hough Transform is one of the most powerful line-detection techniques nowadays and has been widely used in different areas. Unfortunately, its high computation needs often prevent it from being applied in real-time applications without the help of specially designed hardware. In order to solve this problem, a variety of fast implementation algorithms have been proposed. However, none of them can be performed in a real time on an affordable computer. In this paper, we describe a fast implementation of the Hough Transform based on coarse-fine search and the determination of the optimal image resolution. Compared to conventional techniques, our approach decreases the time for needle segmentation by an order of magnitude. Experiments with agar phantom and patient breast biopsy ultrasound (US) image sequences showed that our approach can segment the biopsy needle in real time (i.e., less than 33 ms) on an affordable PC computer without the help of specially designed hardware with the angular rms error of about 1 degrees and the position rms error of about 0.5 mm. <s> BIB011 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> Abstract Rationale and objectives To investigate the potential usefulness of computer-aided diagnosis as a tool for radiologists in the characterization and classification of mass lesions on ultrasound. Materials and methods Previously, a computerized method for the automatic classification of breast lesions on ultrasound was developed. The computerized method includes automatic segmentation of the lesion from the ultrasound image background and automatic extraction of four features related to lesion shape, margin, texture, and posterior acoustic behavior. In this study, the effectiveness of the computer output as an aid to radiologists in their ability to distinguish between malignant and benign lesions, and in their patient management decisions in terms of biopsy recommendation are evaluated. Six expert mammographers and six radiologists in private practice at an institution accredited by the American Ultrasound Institute of Medicine participated in the study. Each observer first interpreted 25 training cases with feedback of biopsy results, and then interpreted 110 additional ultrasound cases without feedback. Simulating an actual clinical setting, the 110 cases were unknown to both the observers and the computer. During interpretation, observers gave their confidence that the lesion was malignant and also their patient management recommendation (biopsy or follow-up). The computer output was then displayed, and observers again gave their confidence that the lesion was malignant and their patient management recommendation. Statistical analyses included receiver operator characteristic analysis and Student t -test. Results For the expert mammographers and for the community radiologists, the A z (area under the receiver operator characteristic curve) increased from 0.83 to 0.87 ( P = .02) and from 0.80 to 0.84 ( P = .04), respectively, when the computer aid was used in the interpretation of the ultrasound images. Also, the A z values for the community radiologists with aid and for the expert mammographers without aid are similar to the A z value for the computer alone (A z = 0.83). Conclusion Computer analysis of ultrasound images of breast lesions has been shown to improve the diagnostic accuracy of radiologists in the task of distinguishing between malignant and benign breast lesions and in recommending cases for biopsy. <s> BIB012 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> RATIONALE AND OBJECTIVES ::: To develop and evaluate a two-stage computerized method that first detects suspicious regions on ultrasound images, and subsequently distinguishes among different lesion types. ::: ::: ::: MATERIALS AND METHODS ::: The first stage of detecting potential lesions was based on expected lesion shape and margin characteristics. After the detection stage, all candidate lesions were classified by a Bayesian neural net based on computer-extracted lesion features. Two separate tasks were performed and evaluated at the classification stage: the first classification task was the distinction between all actual lesions and false-positive detections; the second classification task was the distinction between actual cancer and all other detected lesion candidates (including false-positive detections). The neural nets were trained on a database of 400 cases (757 images), consisting of complex cysts and benign and malignant lesions, and tested on an independent database of 458 cases (1,740 images including 578 normal images). ::: ::: ::: RESULTS ::: In the distinction between all actual lesions and false-positive detections, Az values of 0.94 and 0.91 were obtained with the training and testing data sets, respectively. Sensitivity by patient of 90% at 0.45 false-positive detections per image was achieved for this detection-plus-classification scheme for the testing data set. Distinguishing cancer from all other detections (false-positives plus all benign lesions) proved to be more challenging, and Az values of 0.87 and 0.81 were obtained during training and testing, respectively. Sensitivity by patient of 100% at 0.43 false-positive malignancies per image was achieved in the detection and classification of cancerous lesions for the testing dataset. ::: ::: ::: CONCLUSION ::: The results show promising performance of the computerized lesion detection and classification method, and indicate the potential of such a system for clinical breast ultrasound. <s> BIB013 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> Automatic contouring for breast tumors using medical ultrasound (US) imaging may assist physicians without relevant experience, in making correct diagnoses. This study integrates the advantages of neural network (NN) classification and morphological watershed segmentation to extract precise contours of breast tumors from US images. Textural analysis is employed to yield inputs to the NN to classify ultrasonic images. Autocovariance coefficients specify texture features to classify breasts imaged by US using a self-organizing map (SOM). After the texture features in sonography have been classified, an adaptive preprocessing procedure is selected by SOM output. Finally, watershed transformation automatically determines the contours of the tumor. In this study, the proposed method was trained and tested using images from 60 patients. The results of computer simulations reveal that the proposed method always identified similar contours and regions-of-interest (ROIs) to those obtained by manual contouring (by an experienced physician) of the breast tumor in ultrasonic images. As US imaging becomes more widespread, a functional automatic contouring method is essential and its clinical application is becoming urgent. Such a method provides robust and fast automatic contouring of US images. This study is not to emphasize that the automatic contouring technique is superior to the one undertaken manually. Both automatic and manual contours did not, after all, necessarily result in the same factual pathologic border. In computer-aided diagnosis (CAD) applications, automatic segmentation can save much of the time required to sketch a precise contour, with very high stability. <s> BIB014 </s> Ultrasound image segmentation: a survey <s> B. Breast Cancer <s> We are developing computer vision techniques for the characterization of breast masses as malignant or benign on radiologic examinations. In this study, we investigated the computerized characterization of breast masses on three-dimensional (3-D) ultrasound(US) volumetric images. We developed 2-D and 3-D active contour models for automated segmentation of the mass volumes. The effect of the initialization method of the active contour on the robustness of the iterative segmentation method was studied by varying the contour used for its initialization. For a given segmentation, texture and morphological features were automatically extracted from the segmented masses and their margins. Stepwise discriminant analysis with the leave-one-out method was used to select effective features for the classification task and to combine these features into a malignancy score. The classification accuracy was evaluated using the area A z under the receiver operating characteristic (ROC) curve, as well as the partial area index A z (0.9) , defined as the relative area under the ROC curve above a sensitivity threshold of 0.9. For the purpose of comparison with the computer classifier, four experienced breast radiologists provided malignancy ratings for the 3-D US masses. Our dataset consisted of 3-D US volumes of 102 biopsied masses (46 benign, 56 malignant). The classifiers based on 2-D and 3-D segmentation methods achieved test A z values of 0.87±0.03 and 0.92±0.03, respectively. The difference in the A z values of the two computer classifiers did not achieve statistical significance. The A z values of the four radiologists ranged between 0.84 and 0.92. The difference between the computer’s A z value and that of any of the four radiologists did not achieve statistical significance either. However, the computer’s A z (0.9) value was significantly higher than that of three of the four radiologists. Our results indicate that an automated and effective computer classifier can be designed for differentiating malignant and benign breast masses on 3-D US volumes. The accuracy of the classifier designed in this study was similar to that of experienced breast radiologists. <s> BIB015
Ultrasound imaging of the breast is typically done as an adjunct to physical examination and (X-ray) mammography when breast cancer is suspected BIB001 . Conventional -mode ultrasound does not detect microcalcifications as well as X-ray mammography (a key indicator of cancer) but is used to help distinguish benign masses (cysts and fiboradenomas) from malignant cancerous masses. There have been a number of clinical studies that have looked at assessing how well masses can be characterized from a single image by human observers in terms of visually perceived texture and geometric properties    . This has motivated researchers to look at automating the detection and the characterization of image-derived diagnostic indicators of breast cancer CAD either from a static image frame or image sequences. Thus, the principal segmentation challenges pertain to characterizing the textured appearance and geometry of a cancer relative to normal tissue, and accommodating artefacts such as the possibly strong attenuation across an image and shadowing, as well as the "fuzziness" of cancerous mass boundaries which makes border delineation difficult. Importantly, as we see below, the studies of Stavos et al.    have greatly influenced the design of algorithms for breast mass detection. Interestingly, no significant work has looked at the screening case, i.e., most work has assumed the presence of a, typically single, suspicious mass. Horsch et al. BIB003 presented a method involving thresholding a preprocessed image that has enhanced mass structures. Comparison is made of a partially automatic and fully automatic version of the method with manual delineation on 400 cases/757 images (124 "complex" cysts, 182 benign masses, and 94 malignant masses). They compute four image-based features (shape, echogeneity, margin, and posterior acoustic behavior) defined respectively in terms of the depth-to-width ratio, autocorrelation, "normalized radial gradient," and comparison of gray levels, to test their effectiveness at distinguishing malignant and benign masses. This method was further evaluated in BIB004 and BIB012 to assess the advantages of different features using linear discriminant analysis where the best two features were found to be the depth-to-width ratio (shape) and normalized radial gradient (margin). In later work aimed at further automating the method Drukker et al. BIB005 extended this work to include mass detection by proposing to first filter the images with a radial gradient index filtering technique. The method was tested on the same database as in BIB003 and BIB004 . They showed that 75% of lesions were correctly identified. See also Drukker et al. for further validation on a second database BIB013 . Neural network (NN) based methods have proved to be popular in this area. These aim to make a classification decision based on a set of input features. For instance, Chen et al. BIB006 presented a NN approach where input features were variance contrast, autocorrelation contrast, and the distribution distortion in the (Daubechies) wavelet coefficients and an multilayered perceptron neural (MLP) network with one hidden layer was trained by error backpropagation. The method was applied to a database of 242 cases (161 benign, 81 carcinoma) giving a sensitivity of 98.77% and specificity of 81.77%. They strongly argued that image texture was an important component that made their method successful. Huang and Chen BIB014 proposed an approach that integrates the advantages of NN classification and a watershed-segmentation methods to extract contours of a breast tumor from ultrasound images. The main novelty of this work is in the preprocessing step which helps effectively the watershed algorithm by means of a reasonably good selection of markers. The authors proposed to use a self-organizing map (SOM) texture-based NN in order to select adaptively (i.e., locally) from a set of nine predefined filters the appropriate preprocessing filer to use. Their method was tested on a database of 60 images (21 benign, 39 carinomas), 40 used for training, 20 for testing. Measures of contour difference and area difference between the method and manual delineation were evaluated although strong conclusions cannot be drawn from this evaluation. The above methods treat the segmentation approach as a general image processing problem, i.e., do not take into consideration that the image is an ultrasound image, and do not explicitly consider speckle or imaging artefacts. There are only a few examples of efforts that have attempted to use domain knowledge in breast ultrasound image segmentation. Xiao et al. BIB007 presented an expectation-maximization method that simultaneously estimates the attenuation field at the same time as classification of regions into different (intensity-based) regions. The number of regions (classes) needs to be specified, which in the intended application is not a strong limitation. That method was tested on experimental data with different time gain compensation (TGC) settings to show that their approach gave consistent segmentations under different TGC settings but has not undergone a large clinical assessment. This method is compared to that of Boukerroui in  . Madabhushi and Metaxas BIB008 combined intensity, texture information, and empirical domain knowledge used by radiologists with a deformable shape model in an attempt to limit the effects of shadowing and false positives. (see Fig. 2 for an example result taken from their paper). Their method requires training but in the small database . Using manual delineation of the mass by a radiologist as a reference, and the Hausdorff distance and average distance as boundary error metrics, they showed that their method is independent of the number of training samples, shows good reproducibility with respect to parameters, and gives a true positive area of 74.7%. They also argued that it has automation advantages over the work of Horsch et al. BIB003 . Although some of the previous methods can be applied in 3-D, the literature on 3-D is less extensive. For instance, Chen et al. BIB009 , Chang et al. BIB010   , and Sahiner et al. BIB015 take a deformable active contour approach. Chang et al. BIB010 applied a 3-D version of the "sticks algorithm" BIB002 to reduce speckle noise, followed by 3-D morphological filtering (opening and closing, but details not given) as preprocessing steps. Then they applied an active contour which uses intensity and intensity variance information. The method was tested on eight tumors (four benign, four malignant) with volume estimates compared with estimates by manual delineations. Using the match rate as a performance metric, the average match rate was about 95%. Sahiner et al. BIB015 compared 2-D and 3-D intensity-gradient active contour segmentation based methods, the active contour initialized by hand, and with algorithm parameters determined empirically. Having found the segmentation solution, depth-towidth ratio, a posterior shadowing feature measure, and 72 texture features based on co-occurrence matrix analysis were computed around the boundary for each 2-D slice and linear discriminant analysis used to classify volumes. Four radiologists graded the volumes in terms of perceived malignancy on a scale 1-10. They showed that the radiologist and computer-based methods were not statistically different in classification ( versus average for radiologists). However, they did not look at the accuracy of segmentation in depth and recognized that this was an area of possible improvement. -mode ultrasound imaging can also be used to guide, in real-time, a needle to a mass in biopsies and there has been a few recent papers describing work aimed at automatically tracking the needle during this procedure to localize the position of the mass. The task here is one of contour detection. Ding and Fenster, for instance, assume that the needle is straight and develop an algorithm for needle detection based on the Hough transform BIB011 .