Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> We describe a technique for image encoding in which local operators of many scales but identical shape serve as the basis functions. The representation differs from established techniques in that the code elements are localized in spatial frequency as well as in space. Pixel-to-pixel correlations are first removed by subtracting a lowpass filtered copy of the image from the image itself. The result is a net data compression since the difference, or error, image has low variance and entropy, and the low-pass filtered image may represented at reduced sample density. Further data compression is achieved by quantizing the difference image. These steps are then repeated to compress the low-pass image. Iteration of the process at appropriately expanded scales generates a pyramid data structure. The encoding process is equivalent to sampling the image with Laplacian operators of many scales. Thus, the code tends to enhance salient image features. A further advantage of the present code is that it is well suited for many image analysis tasks as well as for image compression. Fast algorithms are described for coding and decoding. <s> BIB001 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> Sequential search methods characterized by a dynamically changing number of features included or eliminated at each step, henceforth "floating" methods, are presented. They are shown to give very good results and to be computationally more effective than the branch and bound method. <s> BIB002 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> A large number of algorithms have been proposed for feature subset selection. Our experimental results show that the sequential forward floating selection algorithm, proposed by Pudil et al. (1994), dominates the other algorithms tested. We study the problem of choosing an optimal feature set for land use classification based on SAR satellite images using four different texture models. Pooling features derived from different texture models, followed by a feature selection results in a substantial improvement in the classification accuracy. We also illustrate the dangers of using feature selection in small sample size situations. <s> BIB003 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> In the context of the appearance-based paradigm for object recognition, it is generally believed that algorithms based on LDA (linear discriminant analysis) are superior to those based on PCA (principal components analysis). In this communication, we show that this is not always the case. We present our case first by using intuitively plausible arguments and, then, by showing actual results on a face database. Our overall conclusion is that when the training data set is small, PCA can outperform LDA and, also, that PCA is less sensitive to different training data sets. <s> BIB004 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> We present a novel and flexible approach to the problem of feature selection, called grafting. Rather than considering feature selection as separate from learning, grafting treats the selection of suitable features as an integral part of learning a predictor in a regularized learning framework. To make this regularized learning process sufficiently fast for large scale problems, grafting operates in an incremental iterative fashion, gradually building up a feature set while training a predictor model using gradient descent. At each iteration, a fast gradient-based heuristic is used to quickly assess which feature is most likely to improve the existing model, that feature is then added to the model, and the model is incrementally optimized using gradient descent. The algorithm scales linearly with the number of data points and at most quadratically with the number of features. Grafting can be used with a variety of predictor model classes, both linear and non-linear, and can be used for both classification and regression. Experiments are reported here on a variant of grafting for classification, using both linear and non-linear models, and using a logistic regression-inspired loss function. Results on a variety of synthetic and real world data sets are presented. Finally the relationship between grafting, stagewise additive modelling, and boosting is explored. <s> BIB005 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> When large multivariate datasets are analyzed, it is often desirable to reduce their dimensionality. Principal component analysis is one technique for doing this. It replaces the p original variables by a smaller number, q, of derived variables, the principal components, which are linear combinations of the original variables. Often, it is possible to retain most of the variability in the original variables with q very much smaller than p. Despite its apparent simplicity, principal component analysis has a number of subtleties, and it has many uses and extensions. A number of choices associated with the technique are briefly discussed, namely, covariance or correlation, how many components, and different normalization constraints, as well as confusion with factor analysis. Various uses and extensions are outlined. ::: ::: ::: Keywords: ::: ::: dimension reduction; ::: factor analysis; ::: multivariate analysis; ::: variance maximization <s> BIB006 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> Abstract ::: We provide a framework for structural multiscale geometric organization of graphs and subsets of . We use diffusion semigroups to generate multiscale geometries in order to organize and represent complex structures. We show that appropriately selected eigenfunctions or scaling functions of Markov matrices, which describe local transitions, lead to macroscopic descriptions at different scales. The process of iterating or diffusing the Markov matrix is seen as a generalization of some aspects of the Newtonian paradigm, in which local infinitesimal transitions of a system lead to global macroscopic descriptions by integration. We provide a unified view of ideas from data analysis, machine learning, and numerical analysis. <s> BIB007 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> Current diagnosis of prostatic adenocarcinoma is done by manual analysis of biopsy tissue samples for tumor presence. However, the recent advent of whole slide digital scanners has made histopathological tissue specimens amenable to computer-aided diagnosis (CAD). In this paper, we present a CAD system to assist pathologists by automatically detecting prostate cancer from digitized images of prostate histological specimens. Automated diagnosis on very large high resolution images is done via a multi-resolution scheme similar to the manner in which a pathologist isolates regions of interest on a glass slide. Nearly 600 image texture features are extracted and used to perform pixel-wise Bayesian classification at each image scale to obtain corresponding likelihood scenes. Starting at the lowest scale, we apply the AdaBoost algorithm to combine the most discriminating features, and we analyze only pixels with a high combined probability of malignancy at subsequent higher scales. The system was evaluated on 22 studies by comparing the CAD result to a pathologist’s manual segmentation of cancer (which served as ground truth) and found to have an overall accuracy of 88%. Our results show that (1) CAD detection sensitivity remains consistently high across image scales while CAD specificity increases with higher scales, (2) the method is robust to choice of training samples, and (3) the multi-scale cascaded approach results in significant savings in computational time. <s> BIB008 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> The current method of grading prostate cancer on histology uses the Gleason system, which describes five increasingly malignant stages of cancer according to qualitative analysis of tissue architecture. The Gleason grading system has been shown to suffer from inter- and intra-observer variability. In this paper we present a new method for automated and quantitative grading of prostate biopsy specimens. A total of 102 graph-based, morphological, and textural features are extracted from each tissue patch in order to quantify the arrangement of nuclei and glandular structures within digitized images of histological prostate tissue specimens. A support vector machine (SVM) is used to classify the digitized histology slides into one of four different tissue classes: benign epithelium, benign stroma, Gleason grade 3 adenocarcinoma, and Gleason grade 4 adenocarcinoma. The SVM classifier was able to distinguish between all four types of tissue patterns, achieving an accuracy of 92.8% when distinguishing between Gleason grade 3 and stroma, 92.4% between epithelium and stroma, and 76.9% between Gleason grades 3 and 4. Both textural and graph-based features were found to be important in discriminating between different tissue classes. This work suggests that the current Gleason grading scheme can be improved by utilizing quantitative image analysis to aid pathologists in producing an accurate and reproducible diagnosis <s> BIB009 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> Classical shape analysis methods use principal component analysis to reduce the dimensionality of shape spaces. The basic assumption behind these methods is that the subspace corresponding to the major modes of variation for a particular class of shapes is linearised. This may not necessarily be the case in practice. In this paper, we present a novel method for extraction of the intrinsic parameters of multiple shape classes in an unsupervised manner. The proposed method is based on learning the global structure of shape manifolds using diffusion maps. We demonstrate that the method is effective in separating the members of different shape classes after embedding them into a low-dimensional Euclidean space. <s> BIB010 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> The inherent complexity and non-homogeneity of texture makes classification in medical image analysis a challenging task. In this paper, we propose a combined approach for meningioma subtype classification using subband texture (macro) features and micro-texture features. These are captured using the Adaptive Wavelet Packet Transform (ADWPT) and Local Binary Patterns (LBPs), respectively. These two different textural features are combined together and used for classification. The effect of various dimensionality reduction techniques on classification performance is also investigated. We show that high classification accuracies can be achieved using ADWPT. Although LBP features do not provide higher overall classification accuracies than ADWPT, it manages to provide higher accuracy for a meningioma subtype that is difficult to classify otherwise. <s> BIB011 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> Neuroblastoma (NB) is one of the most frequently occurring cancerous tumors in children. The current grading evaluations for patients with this disease require pathologists to identify certain morphological characteristics with microscopic examinations of tumor tissues. Thanks to the advent of modern digital scanners, it is now feasible to scan cross-section tissue specimens and acquire whole-slide digital images. As a result, computerized analysis of these images can generate key quantifiable parameters and assist pathologists with grading evaluations. In this study, image analysis techniques are applied to histological images of haematoxylin and eosin (H&E) stained slides for identifying image regions associated with different pathological components. Texture features derived from segmented components of tissues are extracted and processed by an automated classifier group trained with sample images with different grades of neuroblastic differentiation in a multi-resolution framework. The trained classification system is tested on 33 whole-slide tumor images. The resulting whole-slide classification accuracy produced by the computerized system is 87.88%. Therefore, the developed system is a promising tool to facilitate grading whole-slide images of NB biopsies with high throughput. <s> BIB012 </s> Histopathological Image Analysis: A Review <s> C. Multiscale Feature Extraction <s> We are developing a computer-aided prognosis system for neuroblastoma (NB), a cancer of the nervous system and one of the most malignant tumors affecting children. Histopathological examination is an important stage for further treatment planning in routine clinical diagnosis of NB. According to the International Neuroblastoma Pathology Classification (the Shimada system), NB patients are classified into favorable and unfavorable histology based on the tissue morphology. In this study, we propose an image analysis system that operates on digitized H&E stained whole-slide NB tissue samples and classifies each slide as either stroma-rich or stroma-poor based on the degree of Schwannian stromal development. Our statistical framework performs the classification based on texture features extracted using co-occurrence statistics and local binary patterns. Due to the high resolution of digitized whole-slide images, we propose a multi-resolution approach that mimics the evaluation of a pathologist such that the image analysis starts from the lowest resolution and switches to higher resolutions when necessary. We employ an offine feature selection step, which determines the most discriminative features at each resolution level during the training step. A modified k-nearest neighbor classifier is used to determine the confidence level of the classification to make the decision at a particular resolution level. The proposed approach was independently tested on 43 whole-slide samples and provided an overall classification accuracy of 88.4%. <s> BIB013
Owing to the density of the data and the fact that pathologists tend to employ a multiresolution approach to analyzing pathology data, feature values are related to the viewing scale or resolution. For instance at low or coarse scales color or texture cues are commonly used and at medium scales architectural arrangement of individual histological structures (glands and nuclei) start to become resolvable. It is only at higher resolutions that morphology of specific histological structures can be discerned. In BIB012 , BIB013 , a multiresolution approach has been used for the classification of high-resolution whole-slide histopathology images. The proposed multiresolution approach mimics the evaluation of a pathologist such that image analysis starts from the lowest resolution, which corresponds to the lower magnification levels in a microscope and uses the higher resolution representations for the regions requiring more detailed information for a classification decision. To achieve this, images were decomposed into multiresolution representations using the Gaussian pyramid approach BIB001 . This is followed by color space conversion and feature construction followed by feature extraction and feature selection at each resolution level. Once the classifier is confident enough at a particular resolution level, the system assigns a classification label (e.g., stroma-rich, stroma-poor or undifferentiated, poorly differentiating, differentiating) to the image tile. The resulting classification map from all image tiles forms the final classification map. The classification of a wholeslide image is achieved by dividing into smaller image tiles and processing each image tile independently in parallel on a cluster of computer nodes. As an example, refer to Fig. 11 , showing a hierarchical cascaded scheme for detecting suspicious areas on digitized prostate histopathology slides as presented in BIB008 . Fig. 12 shows the results of a hierarchical classifier for detection of prostate cancer from digitized histopathology. Fig. 12(a) shows the original image with tumor outlined in black. The next three columns show the classifier results at increasing analysis scales. Pixels classified as "nontumor" at a lower magnification (scale) are discarded at the subsequent higher scale, reducing the number of pixels needed for analysis at higher scales. Additionally, the presence of more discriminating information at higher scales allows the classifier to better distinguish between tumor and nontumor pixels. At lower resolutions of histological imagery, textural analysis is commonly used to capture tissue architecture, i.e., the overall pattern of glands, stroma and organ organization. For each digitized histological image several hundred corresponding feature scenes can be generated. Texture feature values are assigned to every pixel in the corresponding image. 3-D statistical, gradient, and Gabor filters can be extracted in order to analyze the scale, orientation, and anisotropic information of the region of interest. Filter operators are applied in order to extract features within local neighborhoods centered at every spatial location. At medium resolution, architectural arrangement of nuclei within each cancer grade can be described via several graph-based algorithms. At higher resolutions, nuclei and the margin and boundary appearance of ductal and glandular structures have proved to be of discriminatory importance. Many of these features are summarized in Tables I and II. D. Feature Selection, Dimensionality Reduction, and Manifold Learning 1) Feature Selection: While humans have innate abilities to process and understand imagery, they do not tend to excel at explaining how they reach their decisions. As such, large feature sets are generated in the hopes that some subset of features incorporates the information used by the human expert for analysis. Therefore, many of the generated features could be redundant or irrelevant. Actually, a large set of features may possibly be detrimental to the classification performance, a phenomenon known as "the curse of dimensionality." Feature selection is a means to select the relevant and important features from a large set of features. This is an increasingly important area of research now that automated quantitative image analysis techniques are becoming more mainstream. Feature selection in histopathological image analysis provides several benefits in addition to improving accuracy. Since images tend to be relatively large, a smaller subset of features needs to be calculated, reducing the computational complexity of classification algorithms. In some applications, it may be preferable to sacrifice the overall performance slightly if this sacrifice greatly reduces the number of selected features. A smaller number of features would also make it easier to explain the underlying model and improve the chances of generalization of the developed system. Additionally, in a multiresolution framework, a set of features proven useful at a given resolution may not be relevant at another resolution, even within the same image. A feature selection algorithm helps determine which features should be used at a given resolution. An optimal feature selection method would require an exhaustive search, which is not practical for a large set of features generated from a large dataset. Therefore, several heuristic algorithms have been developed, which use classification accuracy as the optimality criterion. Well-known feature selection methods include the sequential search methods, namely sequential forward selection (SFS) and sequential backward selection (SBS) BIB002 . SFS works by sequentially adding the feature that most improves the classification performance; similarly, SBS begins with the entire feature set and sequentially removes the feature that most improves the classification performance. Both SFS and SBS suffer from the "nesting effect" whereby features that are selected (SFS) or discarded (SBS) cannot be revisited in a later step and are thus sub-optimal BIB002 . The use of floating search methods, sequential floating forward search (SFFS) and sequential floating backward search (SFBS), in which previously selected or discarded features can be re-evaluated at later steps avoids the nesting problem BIB002 . While these methods still cannot guarantee optimality of the selected feature subset, they have been shown to perform very well compared to other feature selection methods BIB003 and are, furthermore, much more computationally efficient BIB002 . SFFS is one of the most commonly encountered feature selection methods in pathology image analysis literature. More recent feature selection research has focused on such methods as genetic algorithms, simulated annealing, boosting  and grafting BIB005 . A taxonomy of feature selection algorithms is presented in BIB003 . Genetic algorithms and simulated annealing are applications of traditional optimization techniques to feature selection. Boosting, which will be explained in Section VI-C, basically acts as a greedy feature selection process. Grafting (from "gradient feature testing") BIB005 is based on an elegant formulation of the feature selection problem, whereby the classification of the underlying data and the feature selection process are not separated. Within the grafting framework, a loss function is used that shows preference for classifiers that separate the data with larger margins. Grafting also provides an efficient framework for selection of relevant features. Feature selection based on a measure of discriminatory power was proposed in BIB011 , whereby the authors compute the discriminatory power of each of the wavelet packet sub-bands (features) using a dissimilarity measure between approximated probability density functions for different classes. Derived features are then sorted according to the discriminatory power values associated with the corresponding features. 2) Dimensionality Reduction: While feature selection aims to select features (and reduce the feature dimensionality) that best optimize some criterion related to the class labels of the data (e.g., classification performance), dimensionality reduction techniques aim to reduce dimensionality based on some other criterion. Three well-known and commonly used methods of linear dimensionality reduction are principal component analysis (PCA), independent component analysis (ICA), and linear discriminant analysis (LDA). Principal component analysis (PCA) BIB006 looks to find a new orthogonal coordinate system whereby the maximum variance of the data is incorporated in the first few dimensions. Projection of the data onto the individual coordinates encompasses varying degrees of variance; the first coordinate encompasses the largest variance in the data, the second coordinate the next largest variance, and so forth. On the other hand, the LDA is a supervised method; it thus requires class labels for each data sample, mapping the data onto a lower dimensional sub-space that best discriminates data. The goal is to find the mapping, where the sum of distances between samples in different classes is maximized; while the sum of distances between samples in same classes is minimized. LDA can also be formulated in terms of eigenanalysis. A comprehensive discussion of PCA and LDA can be found in BIB004 . Independent component analysis  , looks to find some mixing matrix such that a mixture of the observations (features) are statistically independent. This provides a stronger constraint on the resulting components than PCA, which only requires that the components be uncorrelated. This is why it is particularly well suited for decorrelating independent components from hyperspectral data. Rajpoot and Rajpoot have shown ICA to perform well for extracting three independent components corresponding to three tissue types for segmentation of hyperspectral images of colon histology samples. ICA, however, provides no ranking of the resulting independent components, as does PCA. There are a variety of methods for calculating the independent components (refer to  ), which are generally very computationally intensive. ICA is a higher order method that seeks linear projections, not necessarily orthogonal to each other, as in the case of PCA. 3) Manifold Learning: Recently, nonlinear dimensionality reduction methods have become popular in learning applications. These methods overcome a major limitation of linear dimensionality reduction methods, which assume that geometrical structure of the high-dimensional feature space is linear. In reality, high-dimensional feature spaces comprise of highly nonlinear structures and locality preserving dimensionality reduction methods are highly sought after. Manifold learning is a method of reducing a data set from to dimensions, where while preserving inter-and intra-class relationships between the data. This is done by projecting the data into a low-dimensional feature space in such a way to preserve high dimensional object adjacency. Many manifold learning algorithms have been constructed over the years to deal with different types of data. Graph embedding constructs a confusion matrix describing the similarity between any two images and with feature vectors and , respectively, where and is the total number of images in the data set The embedding vector is obtained from the maximization of the function (5.2) where is the so-called degree matrix, with nonzero values being along the diagonal and . The dimensional embedding space is defined by the eigenvectors corresponding to the smallest eigenvalues of . The value of is generally optimized by obtaining classification accuracies for and selecting the that provided the highest accuracy for each classification task. For image , the feature vector given as input to the graph embedding algorithm produces an -dimensional eigenvector , where is the principal eigenvalue associated with . In BIB009 , a graph embedding algorithm employing the normalized cuts algorithm was used to reconstruct the underlying manifold on which different breast cancer grades were distributed. Manifold learning has also been shown to be useful for shapebased classification of prostate nuclei BIB010 . Rajpoot et al. BIB010 employ diffusion maps BIB007 in order to reduce the dimensionality of shape descriptors down to two dimensions and a fast classification algorithm is derived based on a simple thresholding of the diffusion coordinates.