CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority of European patent application number 16181848.9 filed Jul. 29, 2016, the entire disclosure of which is incorporated by reference herein.

FIELD OF THE INVENTION

The invention relates to a medical imaging apparatus and method for the imaging of a light-sensitive object having a spectral signature, for example biological tissue, in particular light-sensitive tissue of a living body, such as of an eye. The medical imaging apparatus and method in particular are an ophthalmologic apparatus and method, respectively.

BACKGROUND OF THE INVENTION

The observation and imaging of objects such as living tissue is complicated if the object is light-sensitive. Light-sensitivity may cause the tissue to react if illuminated. This reaction may cover characteristics of the tissue that need to be observed to make a diagnosis. In some instances, the illumination may even damage the tissue.

In ophthalmology, the sensitivity of the light-sensitive tissue in the eye results in discomfort for the patient, a higher risk of injury and additional difficulties due to the eye's reflex reaction to light.

To avoid these problems, color cameras are used which produce an acceptable image quality at low light conditions. Such cameras are, however, expensive and bulky and need expensive and large high-quality optics. The large aperture of the optics, which is needed to capture as much of the incoming light as possible, leads only to a limited depth of the field of view.

SUMMARY OF THE INVENTION

The present invention aims to provide a medical imaging apparatus and method which overcomes these problems in the examination of living light-sensitive tissue.

For the medical imaging apparatus initially mentioned, this objective is achieved according to the invention in that the medical imaging apparatus comprises a camera for capturing input images, the spectral signature of the object being represented in the input images by an input set of spectral bands, an output interface for transmitting output images, e.g. to a peripheral device, wherein, in the output images, the spectral signature in the visible light range is represented by an output set of spectral bands, wherein at least one input-only spectral band in the input set is replaced by at least one output-only spectral band in the output set, the medical imaging apparatus further comprising a storage member, in which calibration data are stored for mapping the input set to the output set of spectral bands, and an image processor for determining an intensity in the at least one output-only spectral band depending on the input set and the calibration data.

The medical imaging method according to the invention comprises the steps of acquiring input images in an input set of spectral bands, transmitting output images including an output set of spectral bands, the output set representing the spectral signature in the spectral bands of the output set, converting the input images to the output images by replacing at least one input-only spectral band in the input set by at least one output-only spectral band in the output set and adapting the intensity in the at least one output-only spectral band to the spectral signature using previously stored calibration data.

The invention uses the information which is contained in the input set of spectral bands to determine the spectral signature of the object and to render the spectral signature in the output images using an output set of spectral bands which differs from the input set. This allows to record images in spectral bands at which the object, in particular the sensitive tissue, does not have any sensitivities, e.g. in the IR or NIR range. Using the calibration data, the “true” spectral signature of the object is rendered using the spectral bands of the output set in a spectral range, e.g. the visible light range, which is different from the spectral range in which the input images are captured.

If the output images are displayed on a display the observer would see the very same colors as if observing the tissue in the output set of spectral bands. Thus, imaging is done in a spectral range which is not critical for the tissue e.g. by being invisible to the human eye, whereas displaying is done in the visible-light range to allow the observer proper identification of the tissue by its natural color.

The method and apparatus according to the invention have the advantage of lessened discomfort for the patient and significantly reducing the risk of injury to the tissue. The inventive method and apparatus allow for significantly longer examinations, which helps to prevent errors in diagnosis and therapy.

The apparatus and method may also be used outside medical applications. For example, in military applications, it may be possible to watch an object in ambient light that has only a restricted number of spectral bands with sufficient intensity. Using the invention allows to display the object in its natural colors.

The method and apparatus according to the invention can be further improved by the following features, which are independent of one another with respect to their respective technical effects and which can be combined arbitrarily.

For example, it is preferred that at least two spectral bands of the input set, termed input spectral bands in the following, are in the longer non-visible wavelengths, in particular the IR light range. Especially in ophthalmology, the tissue is significantly less sensitive in the longer non-visible wavelengths compared to the shorter non-visible wavelengths, i.e. UV. However, shorter wavelengths, in blue and or UV spectral bands, could also be used, especially for the study of superficial tissue layers, due to low penetration of shorter wavelength light.

The input images and the output images may be part of a time-series of input images and output images which are captured and/or displayed at a frame rate which is higher than the flicker fusion rate so that any change in the tissue during observation can be smoothly rendered in the output images.

The input images may contain input pixels. The output images may contain output pixels. The input images may be color images. The output images may be color images.

According to one aspect, each input pixel represents a location in the field of view of the camera. Preferably, the output images and input images are congruent and have the same orientation. They may have the same resolution, amount of pixels, size and/or aspect ratio. Preferably, an input pixel in the input image and an output pixel in the output image represent the same, or at least an overlapping, location in the field of view or the same area of the observed tissue, respectively.

The conversion from the input set to the output set is preferably performed on a pixel base: For each pixel in the input image, the corresponding intensities in the output-only spectral bands are computed depending on the intensities in the input spectral bands and the calibration data.

According to another advantageous embodiment of the invention, the medical imaging apparatus may comprise an illumination system for illuminating the object with light including an illumination set of spectral bands. The illumination spectral bands may be bands at least partly in the non-visible light range, in particular in at least the input spectral bands. The output-only spectral band preferably is attenuated or not present in the illumination set. The illumination of the tissue in the input spectral bands ensures a good quality of the input images with respect to depth of field and spatial resolution without the need for special low-light optics or cameras. In the input spectral bands, there is no risk of injuring the tissue or putting the patient in discomfort. Further, in ophthalmological examinations, the eye will not react to an illumination in the non-visible light range by contracting the iris thus allowing for an easier inspection.

In other applications outside medicine, such an illumination may prohibit detection of the illumination source.

Preferably, the illumination is restricted to the non-visible light range. For example, an illumination filter may be provided that restricts illumination of the tissue to the non-visible light range.

As illumination is needed only in the input spectral bands, it is sufficient if the illumination is restricted to the input spectral bands. Thus, the illumination filter can comprise a band-pass filter and/or a band-stop filter system that is transmissive in the input spectral bands.

The illumination may, according to another aspect of the invention, comprise preferably discrete illumination spectral bands which match the input spectral bands of the multispectral camera. The input spectral bands and the illumination spectral bands may be constituted by light having a wavelength of at least 550 nm. In particular, the illumination system may comprise an IR and/or NIR light source.

To avoid irritation of the eye or any other type of interference with the object, the majority, if not all, of the input-only spectral bands is in one embodiment located outside the visible light range, in particular the IR or NIR range, whereas the majority, if not all, of the output-only spectral bands is in the visible light range. The input and/or output spectral bands may be discrete and/or have a substantial overlap. A majority, if not all, of the spectral bands in the input set may be located in the non-visible light range to gather as much information about the spectral signature of the object in the non-sensitive spectral bands as possible. For optimum rendition of the colors of the object, the majority, if not all, of the spectral bands in the output set may be in the visible light range.

The camera may be an imaging spectroscope, in particular a multispectral camera or a hyperspectral camera. The more spectral bands used, the better the spectral resolution of the input spectral signature and the more accurate the assignment of the spectral signature of the object to the output set will be. For some applications a standard color camera e.g. operating the RGB color space may be sufficient.

The output set may comprise in one embodiment only three preferably discrete spectral bands and may in particular be composed by a tri-color color space, such as RGB. Such a color system is sufficient for the output pixels to render the true color of the tissue faithfully for a human observer.

According to another aspect of the invention, the image processor may be configured to transform the input set into the RGB space. Here, the output set comprises the R-, G- and B-spectral bands. Theoretically, this is an ill-posed problem and cannot be solved, because the spectral signature in the input set alone does not contain enough information to determine the color appearance of the tissue in the spectral bands outside the input set. However, prior knowledge of the optical, e.g. reflective, properties of the biological tissue can offer the additional information required to calculate this transformation. This information is contained in the calibration data.

The image processor may be configured to classify the tissue represented by an input pixel depending on the input spectral signature by spectral fitting. The image processor may comprise a spectral analysis module for performing a spectral fitting to the input spectral signature and for computing a tissue reference value from the spectral fitting, wherein the image processor is configured to determine the output spectral signature in dependence of the tissue reference value. Thus, the spectral analysis module may perform a classification of the tissue represented by an input pixel. Classification can be done by assigning the tissue reference value to an individual input spectral signature. This numerical value is then characteristic for a special kind of tissue, e.g. for a tissue having a particular concentration of biological pigments. By determining the output spectral signature in dependence of the tissue reference value, the visual appearance of the tissue in the visible light range can be matched to the tissue having a particular input spectral signature in the non-visible light range. The tissue reference value is representative for the color of the tissue in the visible-light range.

The calibration data may comprise a spectral library. The spectral library may be indexed using the tissue reference value. Each of the multitude of output spectral signatures may be assigned to a unique respective tissue reference value.

The calibration data may comprise parameters of a transfer function which maps the input set onto the output set. Such transfer functions may be determined empirically or analytically, e.g. based on light-tissue interaction models.

The calibration data may comprise parameters of a machine learning system, e.g. a neural network, which was trained with model input and output sets.

The tissue reference value can be anything from a single bit to a bit field, it can be, in a particular instance, a single numerical value or an array of numerical values. It may represent one or more quantities, such as exponents of an exponential Fourier series, or RGB values. The tissue reference value may further be an array of numerical values which indicates the concentration of biological pigments contributing to and/or resulting in the input spectral signature. Such an array can be used to construe the output spectral signature in the visible-light image from model output spectral signatures in a spectral library, where each output spectral signature represents the color of a constituent biological pigment in the visible-light range.

If the concentration of the biological pigments in the tissue is determined automatically from the input spectral signature of the input pixel, e.g. by the above-mentioned spectral fitting, the output spectral signature may be composed from the model output spectral signatures depending on the concentration. The biological pigments of which the concentration is determined and/or which model output spectra are stored in the spectral library, comprises at least one of oxy-hemoglobin, deoxy-hemoglobin and at least one melanin such as eumelanin. Preferably, the number of input spectral bands is at least as large as the number of biological pigments represented in the spectral library.

Instead of such a composition of the output spectral signature from a plurality of model output spectral signatures, the output spectral signatures can also be looked up directly in a differently configured spectral library.

In order to perform the spectral fitting, the storage member may, in another embodiment, comprise representations of model input spectral signatures of certain biological pigments, such as the above, stored for spectral fitting the input spectral signature of the input pixels.

The image processor and any module comprised by the image processor may be constituted by hardware and/or software.

The invention may also relate to a non-transitory computer storage medium storing a program causing a computer to execute the medical imaging method in one of the embodiments described above. The non-transitory computer storage media may be part of the medical imaging apparatus.

In the following, the method and apparatus according to the invention are explained in greater detail with reference to the accompanying drawings, in which exemplary embodiments are shown.

In the Figures, the same reference numerals are used for elements, which correspond to each other in their function and/or structure.

According the description of the various aspects and embodiments, elements shown in the drawings can be omitted if the technical effects of these elements are not needed for a particular application. Vice versa, elements that are not shown or described with reference to the Figures, but are described above, can be added if the technical effect of that particular element is advantageous in a specific application.

