Histopathological Image Analysis: A Review <s> B. Compensating for Tissue Auto-Fluorescence <s> A fully automated approach is presented for robust detection and classification of changes in longitudinal time-series of color retinal fundus images of diabetic retinopathy. The method is robust to: 1) spatial variations in illumination resulting from instrument limitations and changes both within, and between patient visits; 2) imaging artifacts such as dust particles; 3) outliers in the training data; 4) segmentation and alignment errors. Robustness to illumination variation is achieved by a novel iterative algorithm to estimate the reflectance of the retina exploiting automatically extracted segmentations of the retinal vasculature, optic disk, fovea, and pathologies. Robustness to dust artifacts is achieved by exploiting their spectral characteristics, enabling application to film-based, as well as digital imaging systems. False changes from alignment errors are minimized by subpixel accuracy registration using a 12-parameter transformation that accounts for unknown retinal curvature and camera parameters. Bayesian detection and classification algorithms are used to generate a color-coded output that is readily inspected. A multiobserver validation on 43 image pairs from 22 eyes involving nonproliferative and proliferative diabetic retinopathies, showed a 97% change detection rate, a 3 % miss rate, and a 10% false alarm rate. The performance in correctly classifying the changes was 99.3%. A self-consistency metric, and an error factor were developed to measure performance over more than two periods. The average self consistency was 94% and the error factor was 0.06%. Although this study focuses on diabetic changes, the proposed techniques have broader applicability in ophthalmology. <s> BIB001 </s> Histopathological Image Analysis: A Review <s> B. Compensating for Tissue Auto-Fluorescence <s> Two common imaging modalities for histological sections are brightfield and fluorescence microscopy imaging. Hematoxylin-Eosin (H&E) based brightfield microscopy has been the traditional imaging technique for imaging morphology, while an epi-fluorescent microscope is used for immunofluorescent staining of specific proteins or fluorescent in situ hybridization (FISH) for genetic based analysis of DNA. Simultaneous imaging of both microscopy modalities has been difficult due to optical and chemical effects of the H&E dyes. We present a novel sequential imaging and registration technique that enables brightfield and fluorescent imaging on the same tissue section, hence combining the traditional anatomic pathology with the newly emerging field of molecular pathology. First the tissue is labeled with fluorescent biomarkers, and imaged through a fluorescence microscope, and then the tissue is re-labeled with H&E dyes, and imaged again with traditional brightfield. Our robust registration algorithms achieve 99.8% registration success rate on tissue micro array (TMA) sections. <s> BIB002 </s> Histopathological Image Analysis: A Review <s> B. Compensating for Tissue Auto-Fluorescence <s> Image-to-image registration is well-studied, but the challenge of automatically validating registration results remains. In many applications, registration performance is still visually inspected. In Tissue microarrays (TMAs), there are usually hundreds of tissues on a single glass slide, making visual inspection almost impossible. In cases where the TMA slide has to be removed from the microscope and replaced (e.g., in sequential staining, temporal studies, and change analysis), the images on the TMA need to be registered to the corresponding images in the previous steps. The registration has to be robust since undetected registration failure will lead to inaccurate results in subsequent analysis. In this work, we present an automated system to register images of corresponding tissue samples in a TMA, detect registration failures accurately, and automatically re-initialize registration in such cases. The system is image-independent, and reduces the number of cases where the images need to be visually validated. The system has been applied in a study of sequential fluorescent staining of tissue samples on a TMA slide, with 99.88% registration accuracy. <s> BIB003
Tissue auto-fluorescence (AF) is a fundamental problem in microscopy applications, particularly in retrospective studies that use formalin fixed paraffin embedded tissue sections. AF reduces the signal detection sensitivity, and in some cases even causes failure in the detection of fluorescent biomarker signals. In BIB002 , a two-step technique was used to remove the AF from fluorescent microscopy images. Rather than acquiring images of all the dyes at once using a set of optimum filter cubes tuned to specific dyes, the acquisition is done in two steps. In the first step, tissue is stained with only the low AF dyes (e.g., ultraviolet or infrared), and images are acquired using all the filter cubes. Images of these cubes, except the low AF dyes, represent the tissue AF at their specific spectra. In the second step, all the remaining dyes are added, and images of all the cubes are acquired again. Then the first set of images is aligned with the second set using a transformation estimated by registering the low-AF images that are common in both steps. The first step before any AF removal is the correction of the excitation light pattern. The observed image, , can be modeled as a product of the excitation pattern, and the emission pattern. While the emission pattern captures the tissue dependent fluorescent staining, the excitation pattern captures the excitation light. In the logarithm domain, the multiplicative relation can be transformed into a linear form. The excitation pattern can be estimated using the mean of the brightest set of pixels from an ordered set of images (3.1) where denote the ordered pixels ( ), and represents the set of brightest pixels. Assuming that a certain percentage of the image occupies stained tissue (nonzero backgound), is set to an integer to represent this percentage (10% in our experiments). This approximation holds if a large number of images are used in the averaging process. However, a large percentage of pixels are already excluded to eliminate the nontissue pixels in the images. To overcome the limited sampling size, the log of the excitation pattern estimated in (3.1) can be approximated with polynomials. The surface generated by the polynomial coefficients are then used to correct individual images BIB001 . After the images are corrected for their light excitation pattern, the images between the two sequential steps are aligned. From the two common images, one being the reference image from the first step, , and the second being from the subsequent step, , a rigid transformation is obtained, such that the image similarity measure, between and is maximized (3.2) Due to its robustness in registering multimodality images, a mutual information-based image similarity measure is used to deal with tissue loss and folding. Additional robustness is achieved by incorporating the mutual information estimation in a multiresolution framework BIB003 . Once is estimated, all the channels are transformed with this transformation to represent all the images in both acquisitions in the same coordinate system. Then the first set of images is subtracted from the second set of images. To achieve the highest dynamic range, the first set of AF images can be acquired in longer exposure times than the second set of AF-dye mixture images. Fig. 3 shows the acquisition of the Cy3 channel before and after the tissue is stained with Cy3 dye directly conjugated with Estrogen Receptor (ER). The tissue is also stained with a low AF ultraviolet nuclear stain, DAPI (not shown in the figure), which is acquired in both steps and used for aligning the images. The AF removed image is shown in Fig. 3(c) . The arrows point to successfully removed high-AF regions, such as red blood cells. Removing the AF using the proposed two-step approach enables accurate image analysis and quantitation for low abundance proteins and directly conjugated antibodies. 