Histopathological Image Analysis: A Review <s> A. Multimodal Data Fusion/Registration <s> Prostatic adenocarcinoma is the most commonly occurring cancer among men in the United States, second only to skin cancer. Currently, the only definitive method to ascertain the presence of prostatic cancer is by trans-rectal ultrasound (TRUS) directed biopsy. Owing to the poor image quality of ultrasound, the accuracy of TRUS is only 20%-25%. High-resolution magnetic resonance imaging (MRI) has been shown to have a higher accuracy of prostate cancer detection compared to ultrasound. Consequently, several researchers have been exploring the use of high resolution MRI in performing prostate biopsies. Visual detection of prostate cancer, however, continues to be difficult owing to its apparent lack of shape, and the fact that several malignant and benign structures have overlapping intensity and texture characteristics. In this paper, we present a fully automated computer-aided detection (CAD) system for detecting prostatic adenocarcinoma from 4 Tesla ex vivo magnetic resonance (MR) imagery of the prostate. After the acquired MR images have been corrected for background inhomogeneity and nonstandardness, novel three-dimensional (3-D) texture features are extracted from the 3-D MRI scene. A Bayesian classifier then assigns each image voxel a "likelihood" of malignancy for each feature independently. The "likelihood" images generated in this fashion are then combined using an optimally weighted feature combination scheme. Quantitative evaluation was performed by comparing the CAD results with the manually ascertained ground truth for the tumor on the MRI. The tumor labels on the MR slices were determined manually by an expert by visually registering the MR slices with the corresponding regions on the histology slices. We evaluated our CAD system on a total of 33 two-dimensional (2-D) MR slices from five different 3-D MR prostate studies. Five slices from two different glands were used for training. Our feature combination scheme was found to outperform the individual texture features, and also other popularly used feature combination methods, including AdaBoost, ensemble averaging, and majority voting. Further, in several instances our CAD system performed better than the experts in terms of accuracy, the expert segmentations being determined solely from visual inspection of the MRI data. In addition, the intrasystem variability (changes in CAD accuracy with changes in values of system parameters) was significantly lower than the corresponding intraobserver and interobserver variability. CAD performance was found to be very similar for different training sets. Future work will focus on extending the methodology to guide high-resolution MRI-assisted in vivo prostate biopsies. <s> BIB001 </s> Histopathological Image Analysis: A Review <s> A. Multimodal Data Fusion/Registration <s> In this paper we present novel methods for registration and subsequent evaluation of whole mount prostate histological sections to corresponding 4 Tesla ex vivo magnetic resonance imaging (MRI) slices to complement our existing computer-aided diagnosis (CAD) system for detection of prostatic adenocarcinoma from high resolution MRI. The CAD system is trained using voxels labeled as cancer on MRI by experts who visually aligned histology with MRI. To address voxel labeling errors on account of manual alignment and delineation, we have developed a registration method called combined feature ensemble mutual information (COFEMI) to automatically map spatial extent of prostate cancer from histology onto corresponding MRI for prostatectomy specimens. Our method improves over intensity-based similarity metrics (mutual information) by incorporating unique information from feature spaces that are relatively robust to intensity artifacts and which accentuate the structural details in the target and template images to be registered. Our registration algorithm accounts for linear gland deformations in the histological sections resulting from gland fixing and serial sectioning. Following automatic registration of MRI and histology, cancer extent from histological sections are mapped to the corresponding registered MRI slices. The manually delineated cancer areas on MRI obtained via manual alignment of histological sections and MRI are compared with corresponding cancer extent obtained via COFEMI by a novel registration evaluation technique based on use of non-linear dimensionality reduction (locally linear embedding (LLE)). The cancer map on MRI determined by COFEMI was found to be significantly more accurate compared to the manually determined cancer mask. The performance of COFEMI was also found to be superior compared to image intensity-based mutual information registration. <s> BIB002
While digital pathology offers very interesting, highly dense data, one of the exciting challenges will be in the area of multimodal data fusion. One of the big open questions, especially as it pertains to personalized medicine, will be the use of multimodal data classifiers to be able to make therapy recommendations. This will require solving questions both in terms of data alignment and in terms of knowledge representation for fusion of heterogeneous sources of data, in order to answer questions that go beyond just diagnosis, such as theragnosis (therapy prediction) and prognosis. H&E staining is traditionally used for histopathology imaging. Several other modalities exist for imaging of the tissue, each offering its own advantages and limitations. Combining images from different modalities, therefore, may seem to be an attractive proposition, although it does not come without its own challenges, most importantly registration, not to mention the extra cost associated with imaging, storage, and computational time. Registration of image data across the different modalities and fusion of the information contained therein result in a powerful resource of information for diagnosis and prognosis purposes. Fusion methods have been developed for images from different microscopy imaging methods  and micro-scale histopathology and large-scale MR images  -  . Madabhushi et al. BIB001 have been developing computerized detection methods for prostate cancer from high-resolution multimodal MRI . A prerequisite to training a supervised classifier to identify prostate cancer (CaP) on MRI is the ability to precisely determine spatial extent of CaP on the radiological imaging modality. CaP can be precisely determined on whole mount histopathology specimens [ Fig. 17(a) ] which can then be mapped onto MRI [ Fig. 17(b) ]. Fig. 17(c) shows the result of registering [ Fig. 17(b) ] the 2-D MRI slice to the histological section [ Fig. 17(a) ]. This requires the use of sophisticated and robust multimodal deformable registration methods to account for (a) deformations and tissue loss in the whole mount histological specimens during acquisition, and (b) ability to overcome intensity and feature differences between the two modalities (histopathology and MRI). In  , BIB002 a rigid registration scheme called combined feature ensemble based mutual information (COFEMI) was presented that used alternate feature representations of the target and source images to be registered to facilitate the alignment.