FIELD

This disclosure relates generally to pathology.

BACKGROUND

Every year in the United States, twelve million skin lesions are biopsied, with over five million new skin cancer cases diagnosed. After a skin tissue specimen is biopsied, the tissue is fixed, embedded, sectioned, and stained with hematoxylin and eosin (H&E) on (one or several) glass slides, ultimately to be examined under microscope by a dermatologist, general pathologist, or dermatopathologist who provides a diagnosis for each tissue specimen. Owing to the large variety of over 500 distinct skin pathologies and the severe consequences of a critical misdiagnosis, diagnosis in dermatopathology demands specialized training and education. Although inter-observer concordance in dermatopathology is estimated between 90% and 95%, there are some distinctions that present frequent disagreement among pathologists, such as in the case of melanoma vs. benign melanocytic nevi. However, even when diagnosis is accurate, the process can be made more efficient by reducing the turnaround time for each case and by improving pathologist workload distribution. Often, cases get first sent to a generalist—sometimes a dermatologist who diagnoses specimens biopsied in their clinical practice. Only if the diagnosis is not a straightforward one is it sent to a specialist to diagnose. This can result in a delay of days to the patient receiving a diagnosis in sometimes-critical cases. The rise in adoption of digital pathology provides an opportunity for the use of deep learning-based methods for closing these gaps in diagnostic reliability and efficiency.

In recent years, attempts have been made to use deep neural networks for identifying diagnostically relevant patterns in radiology and pathology images. While some attempts appear worth pursuing, the translation of such methods to digital pathology is non-trivial. Among the reasons for this is sheer image size; a whole slide image can contain several gigabytes of image data and billions of pixels. Additionally, non-standardized image appearance (variability in tissue preparation, staining, scanned appearance, presence of artifacts) and the number of pathologic abnormalities that can be observed present unique barriers to development of deployable deep learning applications in pathology. For example, it is known that inter-site variance—in the form of stain and other image properties—can have a strong impact on deep learning models. Nonetheless, deep learning-based methods have recently shown promise in segmentation tasks which can be used to compute features for traditional classifiers, and more recently for some classification tasks. However, many focus only on a single diagnostic class to make binary classifications whose utility breaks down when there is more than one relevant pathology of interest. Additionally, many of these methods have focused on curated datasets consisting of fewer than five pathologies with little diagnostic and image variability.

The insufficiency of known models developed and tested using small curated datasets such as CAMELYON30 has been effectively demonstrated. See Campanella, G. et al.,Clinical-grade computational pathology using weakly supervised deep learning on whole slide images, Nature Medicine1 (2019), hereinafter, “Campanella”. However, while claiming to validate on data free of curation, their dataset features limited capture of not only biological variability (e.g., ignores commonly occurring prostatic intraepithelial neoplasia and atypical glandular structures) but also technical variability originating from slide preparation and scanning characteristics (resulting in exclusion of slides with pen markings, need for retrospective human correction of select results, and poorer performance on externally-scanned images). In contrast to these deep learning systems exposed to contrived pathology problems and datasets, human pathologists are trained to recognize hundreds of morphological variants of diseases they are likely to encounter in their careers and must adapt to variations in tissue preparation and staining protocols. Deep learning algorithms can also be sensitive to image artifacts, in addition to these variations. Some have attempted to account for these issues by detecting and pre-screening image artifacts, either by automatically or manually removing slides with artifacts. Campanella et. al include variability in nonexcluded artifacts which others lack, but still selectively exclude images with ink markings, which have been shown to affect predictions of neural networks.

In addition to the problems identified above regarding automated pathology, problems exist in training and using classifiers due to variations among whole slide images produced by various labs. Because the appearance of a whole slide image varies from lab to lab, scanner to scanner, with the same scanner over time, and even varies based on the brand of stain used, images acquired from different labs or scanners are not effectively interpretable by a model which is trained on the variations of a single lab or image appearance. To the trained human eye, stain differences may be the most dramatic differences between different labs' images. Therefore, much of the work in the field so far has focused on normalizing stain. Typically, stain normalization attempts to extract the two main components of the image color which come from the hematoxylin (H) and eosin (E) stains. There are many methods of stain normalization, and most of the previous gold standard methods have not used any form of deep learning. They typically require a target image, extract the components of that image, and apply a transform computed between those and the components of the image to be normalized. This has several disadvantages:

1. It requires a target or reference image, which is a single example which should represent the “ideal” stain. It is difficult to select an image which has some “ideal” stain properties, and yet exhibits enough variation in tissue types to be useful for comparing images.

2. Methods like Vahadane stain normalization are highly sensitive to the target image, as well as to any background or atypically-colored regions included in either the target or to-be-normalized image. It is therefore easy to get an erroneous/improbable stain normalization result, or one that is not representative.

3. As a result of the above, similar images, e.g., those from slightly separated slices of tissue, will produce inconsistent normalized stain results, which is very problematic when training a model to classify a pathology.

In addition to just changes in the color or stain of the images when compared between labs, other changes in image appearance between labs may be present. Some images had more or less noise, differences in brightness and contrast, hue and saturation, or orientation of the tissue on the slide. While minor changes in these variables might not affect a human's ability to diagnose a case from the images very much, these factors can be important to a deep learning system's ability to correctly identify pathology. There are not currently any methods of correcting for any of these factors in terms of pre-processing images to be assessed by a deep learning system.

SUMMARY

According to various embodiments, a method of stain normalization image processing for digitized biological tissue images is presented. The method includes obtaining a digitized biological tissue image; applying to at least a portion of the digitized biological tissue image an at least partially computer implemented convolutional neural network trained using a training corpus comprising a plurality of pairs of images, wherein each pair of images of the plurality of pairs of images comprises a first image restricted to a lightness axis of a color space and a second image restricted to at least one of: a first color axis of the color space and a second color axis of the color space, whereby the applying causes an output image to be produced; and providing the output image.

Various optional features of the above embodiments include the following. The digitized biological tissue image may include a digitized whole-slide image, and the applying may include applying to the whole-slide image. The providing may include providing to a process comprising an electronic trained classifier, wherein the electronic trained classifier is trained to identify at least one human biological tissue pathology. The digitized biological tissue image may include a digitized cutaneous biological tissue image, and wherein the at least one biological tissue pathology comprise at least one human dermatopathology. Each pair of images of the plurality of pairs of images may include a first image restricted to a lightness axis of a Lab color space and a second image restricted to an axis ‘a’ of the Lab color space and to an axis ‘b’ of the Lab color space. Each second image may be restricted to colors of hematoxylin and eosin. The plurality of pairs of images may include a particular plurality of pairs of images, where each pair of images of the particular plurality of pairs of images comprises a first image that has had noise added to it. The noise may include at least one of hue noise, saturation noise, brightness noise, contrast noise, or intensity noise. The plurality of pairs of images may include a rotated plurality of pairs of images, wherein each pair of images of the rotated plurality of pairs of images comprises a first image that has been rotated by an amount, and a second image that has been rotated by the amount. The training corpus may consist of pairs of images derived from images obtained by a single laboratory.

According to various embodiments, a system for stain normalization image processing for digitized biological tissue images is presented. The system includes at least one electronic processor and at least one persistent electronic memory communicatively coupled to the at least one electronic processor, the at least one persistent memory comprising computer readable instructions that, when executed by the at least one electronic processor, configure the at least one electronic processor to perform operations comprising: obtaining a digitized biological tissue image; applying to at least a portion of the digitized biological tissue image an at least partially computer implemented convolutional neural network trained using a training corpus comprising a plurality of pairs of images, wherein each pair of images of the plurality of pairs of images comprises a first image restricted to a lightness axis of a color space and a second image restricted to at least one of: a first color axis of the color space and a second color axis of the color space, whereby the applying causes an output image to be produced; and providing the output image.

Various optional features of the above embodiments include the following. The digitized biological tissue image may include a digitized whole-slide image, and the applying may include applying to the whole-slide image. The providing may include providing to a process comprising an electronic trained classifier, wherein the electronic trained classifier is trained to identify at least one human biological tissue pathology. The digitized biological tissue image may include a digitized cutaneous biological tissue image, and wherein the at least one biological tissue pathology comprise at least one human dermatopathology. Each pair of images of the plurality of pairs of images may include a first image restricted to a lightness axis of a Lab color space and a second image restricted to an axis ‘a’ of the Lab color space and to an axis ‘b’ of the Lab color space. Each second image may be restricted to colors of hematoxylin and eosin. The plurality of pairs of images may include a particular plurality of pairs of images, wherein each pair of images of the particular plurality of pairs of images comprises a first image that has had noise added to it. The noise may include at least one of hue noise, saturation noise, brightness noise, contrast noise, or intensity noise. The plurality of pairs of images may include a rotated plurality of pairs of images, wherein each pair of images of the rotated plurality of pairs of images comprises a first image that has been rotated by an amount, and a second image that has been rotated by the amount. The training corpus may consist of pairs of images derived from images obtained by a single laboratory.

