Histopathological Image Analysis: A Review <s> B. Immuno-Fluorecence Imaging and Multiple Imaging Modalities <s> The recent development of tissue microarrays-composed of hundreds of tissue sections from different tumors arrayed on a single glass slide-facilitates rapid evaluation of large-scale outcome studies. Realization of this potential depends on the ability to rapidly and precisely quantify the protein expression within each tissue spot. We have developed a set of algorithms that allow the rapid, automated, continuous and quantitative analysis of tissue microarrays, including the separation of tumor from stromal elements and the sub-cellular localization of signals. Validation studies using estrogen receptor in breast carcinoma show that automated analysis matches or exceeds the results of conventional pathologist-based scoring. Automated analysis and sub-cellular localization of beta-catenin in colon cancer identifies two novel, prognostically significant tumor subsets, not detected by traditional pathologist-based scoring. Development of automated analysis technology empowers tissue microarrays for use in discovery-type experiments (more typical of cDNA microarrays), with the added advantage of inclusion of long-term demographic and patient outcome information. <s> BIB001 </s> Histopathological Image Analysis: A Review <s> B. Immuno-Fluorecence Imaging and Multiple Imaging Modalities <s> Accurate spectral decomposition is essential for the analysis and diagnosis of histologically stained tissue sections. In this paper we present the first automated system for performing this decomposition. We compare the performance of our system with ground truth data and report favorable results. <s> BIB002 </s> Histopathological Image Analysis: A Review <s> B. Immuno-Fluorecence Imaging and Multiple Imaging Modalities <s> Temporal and spatial regulation of proteins contributes to function. We describe a multidimensional microscopic robot technology for high-throughput protein colocalization studies that runs cycles of fluorescence tagging, imaging and bleaching in situ. This technology combines three advances: a fluorescence technique capable of mapping hundreds of different proteins in one tissue section or cell sample; a method selecting the most prominent combinatorial molecular patterns by representing the data as binary vectors; and a system for imaging the distribution of these protein clusters in a so-called toponome map. By analyzing many cell and tissue types, we show that this approach reveals rules of hierarchical protein network organization, in which the frequency distribution of different protein clusters obeys Zipf's law, and state-specific lead proteins appear to control protein network topology and function. The technology may facilitate the development of diagnostics and targeted therapies. <s> BIB003 </s> Histopathological Image Analysis: A Review <s> B. Immuno-Fluorecence Imaging and Multiple Imaging Modalities <s> PURPOSE ::: The association hepatocyte growth factor receptor (Met) tyrosine kinase with prognosis and survival in colon cancer is unclear, due in part to the limitation of detection methods used. In particular, conventional chromagenic immunohistochemistry (IHC) has several limitations including the inability to separate compartmental measurements. Measurement of membrane, cytoplasm, and nuclear levels of Met could offer a superior approach to traditional IHC. ::: ::: ::: EXPERIMENTAL DESIGN ::: Fluorescent-based IHC for Met was done in 583 colon cancer patients in a tissue microarray format. Using curvature and intensity-based image analysis, the membrane, nuclear, and cytoplasm were segmented. Probability distributions of Met within each compartment were determined, and an automated scoring algorithm was generated. An optimal score cutpoint was calculated using 500-fold crossvalidation of a training and test data set. For comparison with conventional IHC, a second array from the same tissue microarray block was 3,3'-diaminobenzidine immunostained for Met. ::: ::: ::: RESULTS ::: In crossvalidated and univariate Cox analysis, the membrane relative to cytoplasm Met score was a significant predictor of survival in stage I (hazard ratio, 0.16; P = 0.006) and in stage II patients (hazard ratio, 0.34; P < or = 0.0005). Similar results were found with multivariate analysis. Met in the membrane alone was not a significant predictor of outcome in all patients or within stage. In the 3,3'-diaminobenzidine-stained array, no associations were found with Met expression and survival. ::: ::: ::: CONCLUSIONS ::: These data indicate that the relative subcellular distribution of Met, as measured by novel automated image analysis, may be a valuable biomarker for estimating colon cancer prognosis. <s> BIB004 </s> Histopathological Image Analysis: A Review <s> B. Immuno-Fluorecence Imaging and Multiple Imaging Modalities <s> Two common imaging modalities for histological sections are brightfield and fluorescence microscopy imaging. Hematoxylin-Eosin (H&E) based brightfield microscopy has been the traditional imaging technique for imaging morphology, while an epi-fluorescent microscope is used for immunofluorescent staining of specific proteins or fluorescent in situ hybridization (FISH) for genetic based analysis of DNA. Simultaneous imaging of both microscopy modalities has been difficult due to optical and chemical effects of the H&E dyes. We present a novel sequential imaging and registration technique that enables brightfield and fluorescent imaging on the same tissue section, hence combining the traditional anatomic pathology with the newly emerging field of molecular pathology. First the tissue is labeled with fluorescent biomarkers, and imaged through a fluorescence microscope, and then the tissue is re-labeled with H&E dyes, and imaged again with traditional brightfield. Our robust registration algorithms achieve 99.8% registration success rate on tissue micro array (TMA) sections. <s> BIB005 </s> Histopathological Image Analysis: A Review <s> B. Immuno-Fluorecence Imaging and Multiple Imaging Modalities <s> The systematic study of subcellular location patterns is required to fully characterize the human proteome, as subcellular location provides critical context necessary for understanding a protein's function. The analysis of tens of thousands of expressed proteins for the many cell types and cellular conditions under which they may be found creates a need for automated subcellular pattern analysis. We therefore describe the application of automated methods, previously developed and validated by our laboratory on fluorescence micrographs of cultured cell lines, to analyze subcellular patterns in tissue images from the Human Protein Atlas. The Atlas currently contains images of over 3000 protein patterns in various human tissues obtained using immunohistochemistry. We chose a 16 protein subset from the Atlas that reflects the major classes of subcellular location. We then separated DNA and protein staining in the images, extracted various features from each image, and trained a support vector machine classifier to recognize the protein patterns. Our results show that our system can distinguish the patterns with 83% accuracy in 45 different tissues, and when only the most confident classifications are considered, this rises to 97%. These results are encouraging given that the tissues contain many different cell types organized in different manners, and that the Atlas images are of moderate resolution. The approach described is an important starting point for automatically assigning subcellular locations on a proteome-wide basis for collections of tissue images such as the Atlas. <s> BIB006
Recently, immuno-fluorescent labeling-based image analysis algorithms have been presented to quantify localization of proteins in tissue BIB001 - BIB004 . Commonly used molecular markers are based on chromogenic dyes (such as DAB), or fluorescent dyes (such as Cy dyes or Alexa dyes). Fluorescent dyes have the advantage of multiplexing the dyes to acquire images of multiple proteins. A general overview of molecular labeling, high throughput imaging, and pattern recognition techniques is presented by Price et al.  . With current imaging techniques, it is not possible to simultaneously image H&E dyes and immuno-fluorescent molecular biomarkers due to fluorescent characteristics of the H&E dyes, and due to chemical interactions of H&E dyes with the fluorescently labeled antibodies. Recently, methods have been developed to facilitate sequential imaging and registration techniques that enable different modalities presented digitally from the same histological tissue section. Additionally, sequential imaging and registration enables imaging of multiple immuno-fluorescent stains acquired in multiple steps rather than conventional simultaneous multiplexing techniques. This allows an order of magnitude increase in the number of molecular markers to be imaged for the same tissue section. These techniques make it possible to explore unexamined relationships between morphology, sub-cellular spatial distribution of proteins, and protein-protein interactions. An example of these techniques is shown in Fig. 2 . For brightfield images, hematoxylin stains the nuclei blue [ Fig. 2(a) ], and for fluorescent images DAPI can be used to stain nuclei [blue channel in Fig. 2(a) ]. The first nuclei image is set as the reference image and each of the subsequent nuclei images are registered to the reference. Once the transformation parameters are estimated, then all the channels at a sequential step are mapped onto the reference coordinate system. Fig. 2(b)-(d) shows superimposed b-catenin, pan-keratin, and smooth muscle a-actin markers superimposed on the H&E with green pseudo-color BIB005 . Another recently developed sequential imaging method known as MELC BIB003 has the ability to produce images of the same specimen with up to 100 proteins by utilizing the photo-bleaching characteristics of the fluorescent dyes. One of the major problems with such "multichannel" imaging methods is the registration of the multiplexed images, since physical displacements can easily occur during sequential imaging of the same specimen. In BIB005 , the authors used mutual information-based error metrics to register the nuclei images from sequential staining steps. While the fluorescent images include dedicated nuclei channels (such as DAPI), the nuclei images from the H&E images can be computed using decomposition techniques BIB006 , BIB002 , or using simple ratio or differencing methods that utilize the fact that blue wavelengths are absorbed less than green and red channels by the hematoxylin dye.