Histopathological Image Analysis: A Review <s> A. Object Level Features <s> Image Analysis, a complicated field still in the early stages of application to Pathology, has the capability of rendering major contributions to the diagnosis, prognosis, and management of malignancies of the breast. The present review summarizes the main problems and the general approach to the use of this technique for quantitating immunohistochemical stain results, obtaining DNA histograms, and making de novo diagnoses in routine materials of the Pathology service. In the case of diagnosis, the main steps are sampling, segmentation, and measures of chromatin texture. Currently, the limiting factor for all routine applications of image analysis is probably the absence of a reliable automatic nuclear segmentation. Microsc. Res. Tech. 59:109–118, 2002. © 2002 Wiley-Liss, Inc. <s> BIB001 </s> Histopathological Image Analysis: A Review <s> A. Object Level Features <s> Pancreatic cancer is frequently associated with intense growth of fibrous tissue at the periphery of tumours, but the histopathological quantification of this stromal reaction has not yet been used as a prognostic factor because of the difficulty of obtaining quantitative measures using manual methods. Manual histological grading is a poor indicator of outcome in this type of cancer and there is a clinical need to establish a more sensitive indicator. Recent pancreatic tumour biology research has focused upon the stromal reaction and there is an indication that its histopathological quantification may lead to a new prognostic indicator. Histological samples from 21 cases of pancreatic carcinoma were stained using the sirius red, light-green method. Multiple images from the centre and periphery of each tumour were automatically segmented using colour cluster analysis to subdivide each image into representative colours. These were classified manually as stroma, cell cytoplasm or lumen in order to measure the area of each component in each image. Measured areas were analysed to determine whether the technique could detect spatial differences in the area of each tissue component over all samples, and within individual samples. Over all 21 cases, the area of stromal tissue at the periphery of the tumours exceeded that at the centre by an average of 10.0 percentage points (P < 0.001). Within individual tumours, the algorithm was able to detect significantly more stroma (P < 0.05) at the periphery than the centre in 11 cases, whilst none of the remaining cases had significantly more stromal tissue at the centre than the periphery. The results demonstrate that semi-automated analysis can be used to detect spatial differences in the area of fibrous tissue in routinely stained sections of pancreatic cancer. <s> BIB002 </s> Histopathological Image Analysis: A Review <s> A. Object Level Features <s> The main goal of this dissertation is the development and discussion of techniques for higher-level image analysis, i.e., object-level analysis, of breast cancer imagery. Established cytologic (cell) criteria can be contradictory, and even histologic (tissue) criteria (considered the gold standard for diagnosis) are subject to varied interpretation. There is thus a need to quantitatively define characteristics of breast cancer to bettor coordinate clinical care of women presenting breast masses. We propose here an approach for such quantitative analysis, Quantitative Object- and spatial Arrangement-Level Analysis (QOALA), using expert (pathologist) input to guide the classification process. ::: The main contributions in this work are four-fold. First, quantitatively analyze the utility of multispectral imagery for classification and segmentation tasks in histopathology imagery. Second, we develop object-level segmentations for several histologic classes, as a quantitative object-level segmentation metric. Third, we extract a comprehensive set of both object- and spatial-level features which are used in a feature selection framework for classification of objects and imagery. Fourth, we extend the concepts of object-level features to higher-level image objects, analyze the utility of these high-level objects for image classification, and introduce the concept of a probabilistic graph-based model of imagery. ::: Overall, QOALA yields very good object- and image-level classification performances. More specifically, the object-level features as implemented in QOALA are versatile and general enough to elicit important information from even imperfectly segmented objects. Additionally, the use of non-nuclear features, namely features of cytoplasm and stoma have good classification performance, often exceeding that of nuclei. Higher-level features display a potential to increase both object- and image-level classification performance. <s> BIB003 </s> Histopathological Image Analysis: A Review <s> A. Object Level Features <s> Follicular lymphoma (FL) is a cancer of lymph system and it is the second most common lymphoid malignancy in the western world. Currently, the risk stratification of FL relies on histological grading method, where pathologists evaluate hematoxilin and eosin (H&E) stained tissue sections under a microscope as recommended by the World Health Organization. This manual method requires intensive labor in nature. Due to the sampling bias, it also suffers from inter- and intra-reader variability and poor reproducibility. We are developing a computer-assisted system to provide quantitative assessment of FL images for more consistent evaluation of FL. In this study, we proposed a statistical framework to classify FL images based on their histological grades. We introduced model-based intermediate representation (MBIR) of cytological components that enables higher level semantic description of tissue characteristics. Moreover, we introduced a novel color-texture analysis approach that combines the MBIR with low level texture features, which capture tissue characteristics at pixel level. Experimental results on real follicular lymphoma images demonstrate that the combined feature space improved the accuracy of the system significantly. The implemented system can identify the most aggressive FL (grade III) with 98.9% sensitivity and 98.7% specificity and the overall classification accuracy of the system is 85.5%. <s> BIB004 </s> Histopathological Image Analysis: A Review <s> A. Object Level Features <s> Automated detection and segmentation of nuclear and glandular structures is critical for classification and grading of prostate and breast cancer histopathology. In this paper, we present a methodology for automated detection and segmentation of structures of interest in digitized histopathology images. The scheme integrates image information from across three different scales: (1) low- level information based on pixel values, (2) high-level information based on relationships between pixels for object detection, and (3) domain-specific information based on relationships between histological structures. Low-level information is utilized by a Bayesian classifier to generate a likelihood that each pixel belongs to an object of interest. High-level information is extracted in two ways: (i) by a level-set algorithm, where a contour is evolved in the likelihood scenes generated by the Bayesian classifier to identify object boundaries, and (ii) by a template matching algorithm, where shape models are used to identify glands and nuclei from the low-level likelihood scenes. Structural constraints are imposed via domain- specific knowledge in order to verify whether the detected objects do indeed belong to structures of interest. In this paper we demonstrate the utility of our glandular and nuclear segmentation algorithm in accurate extraction of various morphological and nuclear features for automated grading of (a) prostate cancer, (b) breast cancer, and (c) distinguishing between cancerous and benign breast histology specimens. The efficacy of our segmentation algorithm is evaluated by comparing breast and prostate cancer grading and benign vs. cancer discrimination accuracies with corresponding accuracies obtained via manual detection and segmentation of glands and nuclei. <s> BIB005
Fundamentally, object-level analysis depends greatly on some underlying segmentation mechanism. It is the segmentation methodology that determines what constitutes an object. Commonly, an object is defined as a connected group of pixels satisfying some similarity criterion. The main focus is often on the segmentation of nuclei; there exists little work that explicitly uses features of cytoplasm and stroma, although some researchers have hinted at the need for such features BIB002 , BIB001 . Preliminary work BIB003 has demonstrated the feasibility of other histologic features for image classification in H&E stained breast cancer. Madabhushi et al.  used cytoplasmic and stromal features to automatically segment glands in prostate histopathology. Moreover, it appears that histologic objects may not need to be perfectly segmented to be properly classified when a list of comprehensive features is used in a feature selection framework BIB003 . Classification performance in distinguishing between different grades of prostate cancer was found to be comparable using manual and automated gland and nuclear segmentation  . These results suggest that perfect segmentation is not necessarily a prerequisite for good classification. Object-level features can be categorized as belonging to one of four categories: size and shape, radiometric and densitometric, texture, and chromatin-specific. While the radiometric and densitometric, texture, and chromatin-specific features could be considered low-level features that can be extracted from local neighborhoods, the size and shape metrics are true object-level metrics. A summary of object-level features is listed in Table I ; definitions for all listed features can be found in BIB003 . These features were compiled from a comprehensive literature search on cytopathology and histopathology image analysis. In addition, various statistics measures for any of the vector quantities are also commonly calculated. Thus, the mean, median, minimum, maximum, standard deviation, skewness, and kurtosis can be calculated for all vector features. For an RGB image, all relevant features are extracted for each individual color channel; hence, the total number of object-level features can easily exceed 1000 for the list of features in Table I . It should be noted that these features are most commonly extracted from high-resolution imagery (see next section), but are relevant for any resolution. An approach that semantically describes histopathology images using model based intermediate representation (MBIR) and incorporates low-level color texture analysis was presented in BIB004 . In this approach, basic cytological components in the image are first identified using an unsupervised clustering in the La*b* color space. The connected components of nuclei and cytoplasm regions were modeled using ellipses. An extensive set of features can be constructed from this intermediate representation to characterize the tissue morphology as well as tissue topology. Using this representation, the relative amount and spatial distribution of these cytological components can be measured. In the application of follicular lymphoma grading, where the spatial distribution of these regions varies considerably between different histological grades, MBIR provides a convenient way to quantify the corresponding observations. Additionally, low-level color texture features are extracted using the co-occurrence statistics of the color information. Due to the staining of the tissue samples, the resulting digitized images have considerably limited dynamic ranges in the color spectrum. Taking this fact into account, a nonlinear color quantization using self-organizing maps (SOM) is used to adaptively model the color content of microscopic tissue images. The quantized image is used to construct the co-occurrence matrix from which low-level color texture features are extracted. By combining the statistical features constructed from the MBIR with the low-level color texture features, the classification performance of the system can be improved significantly. Fig. 6 shows some of the textural image features for discriminating between benign breast epithelial tissue BIB005 (DCIS, Fig. 6(a) ) and DCIS ( Fig. 6(d) ). Fig. 6(b) , (e), shows the corresponding Gabor filter responses while Fig. 6(c), (f) shows the corresponding Haralick feature images.