FIELD OF THE INVENTION

The present invention generally relates to the field of medical diagnostics. More particularly, the present invention pertains to improved systems and methods for processing digital microscope images to facilitate detection of cancerous and pre-cancerous tissue and cells.

BACKGROUND OF THE INVENTION

Pathologists typically utilize high-resolution microscopes to examine tissue samples, for example, to identify signs of cancer or pre-cancerous cells. In order to make an accurate and correct diagnosis, the pathologist must see cellular and tissue features in focus under a high-resolution microscope. However, high-resolutions microscopes used by pathologists have limitations which make it difficult to analyze thick biological specimens that have objects of interest on different planes.

Specifically, a microscope's lens can only be focused at single point, and there is a finite distance in front of and behind this focal point that may be considered sharp. This finite distance is known as the depth of field. As is well known, high-resolution microscopes, such as those used by pathologists, have a limited or narrow depth of field. As a result, objects that appear outside of a given depth of field or focal plane of the microscope are blurred and out of focus, forcing the pathologist to manually and continually alter the focus when viewing a thick sample. This limits the productivity of the pathologist and also increases the likelihood he or she will miss a subtle feature that may appear only in a narrow focal plane.

This limitation is particularly acute in the analysis of thick tissue specimens (e.g., those that are thicker than the depth of field of a microscope objective) or uneven tissue specimens, since the entire specimen cannot be imaged in a single focal plane. The three-dimensional character of such specimens requires constant refocusing to observe cells at various contours of the sample. As a result, the pathologist does not see the whole sample in focus, limiting the pathologist's ability to recognize subtle diagnostic features that expand over several focal planes.

For example, when obtaining a non-lacerational brush biopsy of a tissue, a brush is used that is sufficiently stiff so as to penetrate tissue. In the process of obtaining a full thickness tissue specimen, tissue fragments in addition to single cells and cell clusters are obtained and transferred onto a microscope slide. The collection of such thick specimens is described, for example, in U.S. Pat. No. 6,258,044.

Such specimens contain single cells, cell clusters and tissue fragments, and are essentially a hybrid between a cytological smear and histological sections. Such specimens may be, for example, 20 to 60 microns thick. However, the depth of field of a typical 20× microscope with a 0.75 NA (Numerical Aperture) may be just 4 microns. Thus, such specimens cannot be readily imaged and, as a result, conventional microscopy does not present all of the information that a pathologist needs when making a diagnosis (e.g., an image that is entirely in focus).

The ability to view tissue fragments, in addition to single cells, would confer an advantage to a pathologist in making a diagnosis. For example, intact tissue provides the pathologist with important information about a tissue's architecture that is not available in cytological smears. This benefit is especially critical in the evaluation of gastrointestinal tissue, which is a complex tissue containing various cell types including, for example, glandular, squamous and columnar epithelium.

One solution to the above problem is provided in U.S. Pat. No. 8,199,997 (“the '997 patent”). That patent discloses systems and methods that compose a two-dimensional image out of a thick, three-dimensional, specimen. This allows a pathologist to capture the information available from a three-dimensional specimen without the drawbacks associated with a conventional microscope. The systems and methods disclosed in the '997 patent utilize extended depth of focus (“EDF”) processing techniques. As described therein, with EDF processing, an automated microscope captures a set of image slices taken at regular intervals along the z-axis (at the same location) and then recovers from each slice those pixels that are in focus to build a single composite image from the in focus pixels.

Although the invention of the '997 patent represents a significant improvement over conventional microscopy techniques, further improvements in EDF-based imaging systems are needed for imaging thick, semi-transparent biological specimens. In this regard, conventional EDF systems and methods bring all image elements into sharpness, regardless of their location in a set of images. These systems and methods may do this by iteratively traversing a collection of images and identifying the sharpest portions of each image. A composite image is then formed using only the pixels located in the sharpest portions of each image in the image set. This conventional process works well with non-transparent or opaque objects, where it is not possible to see any objects below the surface. However, in semi-transparent images objects, such as tissue samples, objects below the surface are visible to the microscope, which introduces additional complexity.

Specifically, conventional EDF systems and methods would bring objects on the surface and those underneath the surface into focus, making it appear that both the upper and lower objects are on the same focal plane and causing objects to appear closer to each other than they really are. Such undesirable image artifacts can cause cells to look crowded and therefore unhealthy, which could significantly change the diagnosis of the area rendered by the pathologist and/or computer system, e.g., from benign to dysplastic (i.e., pre-cancerous). In this regard, healthy tissue will appear to have regular spacing between the cells, whereas in cancerous tissue the spacing between cells is highly irregular, or cells are not uniformly aligned with one another. Conventional EDF also has a natural tendency of decimating the z-relationship between objects to enhance focus, whereas it would be desirable to preserve the spacing between nuclei, and thereby the true diagnosis.

Accordingly, systems and methods are needed that create a composite image from a collection of images while preserving the spatial relationship between objects on different planes. Further, systems and methods are needed that identify the optimal focal plane for a collection of images.

Another problem with conventional EDF systems is that the magnification changes as the microscope's objective is moved up and down between focal planes. In particular, when the objective is moved, new objects will appear in focus, while other objects will becomes less focused. At the same time, those less focused objects also become smaller due to magnification changes. As a result, the edges in the image move as the focus changes and the image correspondingly shrinks, potentially causing the algorithm to recognize each moving edge as a separate edge at each focal plane. This may result in the system splitting a single edge into a “stair-case” of multiple adjacent edges in the composite EDF image. In such circumstances, the “moving” false edges may overwhelm the image itself and introduce staircase artifacts on the composite EDF images. These artifacts may, for example, appear as white flakes on the composite image.

In EDF systems, it is often advantageous to obtain a stack of images along a z-axis with large steps between images (e.g., for increased speed in the imaging process). This step size may be close to the depth of field of the system. Therefore, systems and methods are needed that preserve object edges, yet allow large steps between images.

Furthermore, systems and methods are needed that take advantage of the valuable diagnostic information uniquely contained in a brush biopsy sample. By way of example, a large number of patients in the United States and across the globe undergo endoscopy procedures, whereby a doctor observes sections of the upper gastrointestinal tract, the bile duct or other areas of the body using an endoscope. In such procedures, a doctor may perform forceps biopsies and/or brush biopsies to retrieve tissue samples for laboratory analysis.

During a forceps biopsy procedure, small sections of tissue are excised from focused areas of the esophagus at given intervals. In a laboratory, the excised tissue segments are sliced with a microtome into flat sheets for analysis by a pathologist. As such, a pathologist reviewing these conventional tissue specimens analyzes substantially flat tissue sections where minimal refocusing is necessary.

During a brush biopsy procedure, on the other hand, a brush biopsy instrument having stiff bristles is used to sweep a wide area of tissue and obtain a full thickness sample of tissue of the wide tissue area. The biopsy brush removes small tissue segments that are transferred to a specimen slide substantially intact. Since these tissue segments are not sliced (as described above with respect to forceps biopsies), the natural architecture of the tissue is maintained. Significantly, this preserves the en face view of the tissue for observation by a pathologist and/or analysis by a computer system (unlike conventional histologic tissue preparations where the en face view is destroyed due to tissue slicing).

The en face view of the tissue confers valuable diagnostic information. For example, cells of the gastrointestinal tract are organized in a lattice structure that forms a “honeycomb.” This hexagonal tissue architecture is typical of glandular cells in the body, such as bile duct, colon, breast, etc., and of transitional regions where squamous and glandular tissues meet such as esophageal or endocervical cells.

In healthy tissue, evenly-spaced nuclei can be observed forming the honeycomb appearance. However, in early dysplasia, individual nuclei may become slightly enlarged and the normal nuclear cytoplasmic ratio increases. When this occurs, neighboring nuclei grow closer to one another and begin crowding together. In addition, instead of packing into an organized honeycomb, the nuclei become disorganized and the relationships of cells to each other become haphazard in nature. Thus, the presence or absence of a honeycomb structure and the degree of crowding and disorganization are important diagnostic features in detecting early stage disease and discerning between dysplastic and benign conditions.

Although brush biopsies are able to obtain tissue fragments that retain the en face view of the tissue and retain the honeycomb for clinical observation, constituent cells that form the honeycomb are often located in different focal planes, and as such, a pathologist is unable to observe the honeycomb in-focus. Rather, the pathologist is required to view one or more cells at a first focal distance in isolation, then view one or more cells at a second focal distance in isolation, and so on. Not only is this manual process tedious, it is also unreliable. In this regard, the pathologist must remember the relationship and distance between the cells at the different focal distances and then mentally piece together all of the information that he or she has observed. By way of analogy, rather than viewing a picture of a forest, the pathologist is forced to look at individual trees and try to construct in his/her mind an image of the forest.

Known EDF techniques have not resulted in the honeycomb being properly imaged, as they fail to preserve the spatial relationship between cells in thick, semi-transparent samples. As a result, the honeycomb is not clearly imaged and its regularity and potential abnormality is harder to evaluate. In conventional EDF systems, all the nuclei will appear on the same plane, thereby making the honeycomb appear more crowded, giving the pathologist the false impression that the cells are becoming dysplastic.

Accordingly, improved systems and methods are needed that can create an in-focus view of the honeycomb structure for analysis by a computer and/or a pathologist.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an EDF system that generates an in-focus composite image of a biological sample whereby diagnostically important image objects are presented in focus and underlying objects are de-emphasized.

It is another object of the invention to determine a plurality of optimal focal planes for different segments of the biological sample and obtain image objects from the plurality of the optimal focal planes to generate a digital composite image.

It is yet another objection of the invention to determine whether a distance between cells in a tissue are within a predetermined healthy distance, and when such determination is made, to de-emphasize image objects underlying a plane occupied by the cells within the predetermined healthy distance.

It is another object of the invention to generate an en face image of a tissue where constituent cells comprising the tissue are located on different planes.

