Convolutional neural networks (CNNs) have been widely used to achieve the state-of-the-art results for different histology image analysis tasks such as nuclei detection and classification [1]}, [2]}, [3]}, metastasis detection [4]}, [5]}, [6]}, tumor segmentation [7]} and cancer grading [8]}, [9]}, [10]}. Each task requires a different amount of contextual information, for instance, cell classification needs only high-resolution cell appearance along with little neighboring tissue whereas tumour detection and segmentation rely on a larger context covering multiple cells simultaneously. Due to tumour heterogeneity, cancer grading requires high-resolution cell information as well as the contextual spatial organization of cells in the tumour microenvironment (TME). Most existing CNN based methods applied to histology images follow a patch based approach to training different models tends to ignore contextual information due to memory constraints. Although these models are often trained on a large number of image patches extracted from histology images, often spatial relationships between neighbouring patches are ignored. Due to the lack of large contextual information, the inference is independent of underlying tissue architecture and it is performed based on the limited context captured by individual patches. This approach works well for problems where contextual information is relatively less important for prediction. However, contextual information becomes vital in problems where diagnostic decisions are made on the basis of underlying tissue architecture such as cancer grading.
