CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. national phase application of International Application No. PCT/EP2018/056475 filed on Mar. 15, 2018, which claims the benefit of EP Application Serial No. 17161811.9 filed on Mar. 20, 2017 and is incorporated herein by reference.

FIELD OF THE INVENTION

The invention relates in magnetic resonance imaging, in particular to image segmentation in magnetic resonance imaging.

BACKGROUND OF THE INVENTION

A large static magnetic field is used by Magnetic Resonance Imaging (MRI) scanners to align the nuclear spins of atoms as part of the procedure for producing images within the body of a patient. This large static magnetic field is referred to as the B0 field.

During an MRI scan, Radio Frequency (RF) pulses generated by a transmitter coil cause perturbations to the local magnetic field, and RF signals emitted by the nuclear spins are detected by a receiver coil. These RF signals are used to construct the MRI images. These coils can also be referred to as antennas. Further, the transmitter and receiver coils can also be integrated into a single transceiver coil that performs both functions. It is understood that the use of the term transceiver coil also refers to systems where separate transmitter and receiver coils are used. The transmitted RF field is referred to as the B1 field.

MRI scanners are able to construct images of either slices or volumes. A slice is a thin volume that is only one voxel thick. A voxel is a small volume over which the MRI signal is averaged, and represents the resolution of the MRI image. A voxel may also be referred to as a pixel herein.

United States patent application US 2008/0077006 A1 discloses an MR method for determining local relaxation time values of an examination object. Firstly, two or more MR images of the examination object are recorded, each with different time parameter sets of an imaging sequence. MR images of a phantom are likewise recorded, wherein the same time parameter sets of the imaging sequence are used and wherein the phantom has a known spatial distribution of relaxation time values. The local relaxation time values of the examination object are determined by comparing image values of the MR images of the examination object with image values of the MR images of the phantom and by assigning image values of the MR images of the phantom to relaxation time values according to the known spatial distribution of relaxation time values of the phantom.

The US-patent application US2011/0188720 is concerned with the problem of providing a more accurate and reliable automated segmentation of intensity images. This US-patent application proposes to achieve this by scaling the contrast of the intensity image on the basis of the (MR) acquisition parameters.

SUMMARY OF THE INVENTION

The invention provides for a medical imaging system, a computer program product and a method in the independent claims. Embodiments are given in the dependent claims.

After acquisition of a magnetic resonance image, the magnetic resonance image may be segmented or divided into different regions. This may be useful to identify anatomical structures or even to identify anatomical abnormalities such as tumors. A difficulty with using automatic segmentation algorithms is that differences in the contrast or grey values in the image for particular tissue types may vary according to changes in the acquisition protocol. Embodiments may solve this problem by using metadata that contains reference grey scale value data for a particular image. The reference grey scale value data may be used to modify the segmentation algorithm.

In one aspect, the invention provides for a medical imaging system comprising a memory containing machine-executable instructions. The medical imaging system further comprises a processor. Execution of the machine-executable instructions cause the processor to receive a magnetic resonance image. The magnetic resonance image may for example be received from another computer system or medical imaging system or may be generated by the medical imaging system itself by reconstructing the magnetic resonance image.

Execution of the machine-executable instructions further causes the processor to receive meta data descriptive of the magnetic resonance image. The meta data comprises reference grayscale value data for two or more tissues. The reference grayscale value data may be descriptive of the grayscale value of tissues imaged within the magnetic resonance image. Execution of the machine-executable instructions further cause the processor to segment the magnetic resonance image using an image segmentation algorithm using the reference grayscale value data. The reference grayscale value data may be used to modify the behavior of the image segmentation algorithm. This embodiment may be beneficial because magnetic resonance images or data from different MRI systems and/or different imaging protocols may have different grayscale values for identical types of tissues.

Modifying the image segmentation algorithm with the reference grayscale value data may enable more accurate segmentation of a magnetic resonance image.

In different examples, the reference grayscale value data may take different forms. In one example it may comprise a set of reference grayscale values. That is to say there may be numerical values which are referenced to a particular tissue type of interest. These reference grayscale values may for example either be modeled or may be experimentally measured. For example, phantoms may be measured to obtain a reference grayscale value. In yet other cases, the data can be measured in the form of an image or even a histogram for a particular region. In this case, the segmentation of the magnetic resonance imaging using the image segmentation algorithm may also comprise processing the grayscale value data into a form which can be used to modify the image segmentation algorithm. For example if the reference grayscale value data is provided in the form of an image there may be some data processing or statistical measures which are applied to that image or the region within the image.

In another embodiment, the image segmentation is a trained image segmentation algorithm.

In another embodiment, execution of the machine-executable instructions further cause the processor to render the magnetic resonance image with the segmentation.

In another embodiment, the medical imaging system further comprises a magnetic resonance imaging system. The medical imaging system further comprises pulse sequence commands for controlling the magnetic resonance imaging system to acquire magnetic resonance data. Execution of the machine-executable instructions further causes the medical imaging system to control the magnetic resonance imaging system using the pulse sequence commands to acquire the magnetic resonance data. Execution of the machine-executable instructions further causes the medical imaging system to reconstruct the magnetic resonance image from the magnetic resonance data. This reconstruction may be equivalent to receiving the magnetic resonance image. Magnetic Resonance (MR) data is defined herein as being the recorded measurements of radio frequency signals emitted by atomic spins using the antenna of a magnetic resonance apparatus during a magnetic resonance imaging scan. Magnetic resonance data is an example of medical imaging data. A Magnetic Resonance (MR) image is defined herein as being the reconstructed two or three dimensional visualization of anatomic data contained within the magnetic resonance imaging data.

Execution of the machine-executable instructions further causes the medical imaging system to generate the reference grayscale value data. This may also include attaching or including the reference grayscale value data into the meta-data. Generating the reference grayscale value data may be equivalent to receiving the metadata with the reference grayscale value data.

The reference grey scale data may take different forms in different examples. The reference grey scale data in some examples may be generated by modeling and in other examples it may be taken from measurements from phantoms. In addition to being provided in different way the reference grey scale data may take different forms. The reference grey scale data could include raw image data for processing, it could include a summary of grey scale values for particular tissue types, and/or even comprise statistics or summaries of grey scale values for tissue types or organs. An example would be a histogram of grey scale values for a phantom image or a simulated grey scale histogram.

In some examples as mentioned above, the reference grayscale value data may for instance be generated by taking measurements from phantoms. In this case there may be multiple regions of interest which are imaged with the pulse sequence commands. The magnetic resonance data may therefore be used to reconstruct more than one single image. For example there may be a region of interest which corresponds to a clinical region of interest of the subject and one or more region of interest to image phantoms. Likely in this case the same pulse sequence commands would be used for all of the various regions of interest however, the area being imaged would be changed. That is to say the same pulse sequence commands are used from the different regions of interest but that the region that is being imaged is simply shifted or moved.

In another embodiment, the generation of the reference grayscale value data is performed by modeling using the pulse sequence commands and a magnetic resonance simulator. This embodiment may be beneficial because it may provide a straight forward means of generating the reference grayscale value data.

In another embodiment, the magnetic resonance simulator is a Bloch simulator.

In another embodiment, the magnetic resonance simulator is a phase graph simulator.

In another embodiment, execution of the machine-executable instructions further causes the processor to identify multiple phantom regions corresponding to each of the two or more tissue types within a magnetic resonance image. In some embodiments the magnetic resonance image may be a single magnetic resonance image. In other examples there may be a diagnostic magnetic resonance image and a phantom magnetic resonance image. If there are two images then the multiple phantom regions will be within the phantom magnetic resonance image. The reference grayscale value data is at least partially determined using the magnetic resonance image. This embodiment may be beneficial because it may provide a means to directly measure an expected gray value.

The determinant of the reference grayscale value data in this matter may take different forms. For example it may be generated by calculating a statistical value of grayscales within each of the multiple phantom regions. In other cases it may just be a portion of the raw image data which can then be analyzed later.

In another embodiment, the magnetic resonance imaging system comprises an imaging coil. The imaging coil comprises a magnetic resonance phantom for each of the two or more tissues types. This embodiment may be beneficial because it may provide an easy to use and uniform way of providing the reference grayscale value data.

In another embodiment, the magnetic resonance phantom for each of the two or more tissue types has a predetermined geometric location relative to each other. The multiple phantom regions are identified at least partially using the predetermined geometric location. This embodiment may be beneficial because it may make it easier to identify the multiple phantom regions corresponding to the two or more tissues types.

In another embodiment the phantom for each of the two or more tissue types comprises a phantom material with a controlled proton density, a controlled T1 value, and a controlled T2 value which models the corresponding two or more tissue types. This embodiment may be beneficial because controlling the proton density, T1 and T2 values may be an effective means of modeling the grayscale of a tissue using magnetic resonance imaging.

In another embodiment, the image segmentation algorithm comprises an atlas-matching algorithm for segmenting the magnetic resonance image. The atlas-matching algorithm is adjusted using the reference grayscale values. This embodiment may be beneficial because using the reference grayscale values with the atlas-matching algorithm may enable accurate segmentation results on a broad scale of acquisition protocols and different MR systems. When using an atlas-matching algorithm to segment the magnetic resonance image, the reference grey scale values can be either used to modify the behavior of the atlas-matching algorithm or to modify the magnetic resonance image so that the segmentation is performed properly.

In another embodiment, the image segmentation algorithm comprises a mesh model for segmenting the magnetic resonance image. The mesh model is adjusted using the reference grayscale values. This for example may be accomplished by identifying portions of the mesh model that use grayscale values. For example the reference grey scale values may be useful in modifying the behavior of edge detection algorithms or when evaluating grey scale values in the magnetic resonance image near the boundaries of the model. This may provide for a more robust means of operating the mesh model.

The mesh model may for example be a parametric mesh model or a deformable mesh model that minimizes a penalty function when adapting the mesh model.

In another embodiment, the image segmentation algorithm comprises a voxel classifier for segmenting the magnetic resonance image by classifying individual voxels using gray scale values and a neighborhood of the individual voxels. The voxel classifier algorithm may classify each voxel as to a particular tissue type and/or classify it as belonging to a particular organ or anatomical structure. The voxel classifier may use grey scales as input. The voxel classifier may also use voxels within a predetermined neighborhood of a voxel to assist in the classification. The voxel classifier may be adjusted using the reference gray scale values. The voxel classifier could for example be a histogram-based segmentation model or a neural network segmentation model.

In another embodiment, the image segmentation algorithm comprises a histogram-based segmentation model for segmenting the magnetic resonance image. The histogram-based segmentation model is adjusted using the reference grayscale values. For example either the grayscale values in the magnetic resonance image or in the histogram model itself could be adjusted by comparing the reference grayscale values to the grayscales in the histogram-based segmentation model.

In another embodiment, the image segmentation algorithm comprises an edge detection algorithm for approximating boundary detections between the two or more tissues types. The reference gray scale values may be used, for example, to decide whether boundaries between two types of tissue or organs change from bright to dark or vice versa as the boundary between the two is crossed. The reference grey scale values can be used to decide the appropriate gradient direction of the grey scales as the boundary between tissue types is crossed. The reference grey scale values may also be used to determine the grey scale values within an organ or anatomical region once the boundary has been crossed. This may be useful in reducing the chances that a boundary is incorrectly identified by the model.

In another embodiment, the image segmentation algorithm comprises a generalized Hough transform for detecting anatomical structures corresponding to the two or more tissue types within the magnetic resonance image. The generalized Hough transform comprises a boundary detection algorithm. Execution of the machine-executable instructions comprises modifying the boundary detection algorithm using the reference grayscale value data before segmenting the magnetic resonance image.

In another embodiment, the image segmentation algorithm comprises a random forest decision algorithm for detecting anatomical structures corresponding to the two or more tissue types within the magnetic resonance image. The random forest decision algorithm could possibly contain gray scale values which are used and may also comprises a boundary detection algorithm. Execution of the machine-executable instructions comprises modifying the boundary detection algorithm using the reference grayscale value data before segmenting the magnetic resonance image.

In another embodiment, the image segmentation algorithm comprises a parametric mesh model. The generalized Hough transform is configured for initializing the parametric mesh model. The parametric model is configured for adapting the parametric mesh model to segment the magnetic resonance image. In a variation of this embodiment the random forest decision algorithm is used to initialize the parametric mesh model instead.

In another embodiment, the parametric mesh model comprises boundary points. The parametric mesh model could for example be trained for individually adjusting each boundary point.

In another embodiment execution of the machine-executable instructions further cause the processor to rescale gray values in the magnetic resonance image to a reference interval. Execution of the machine-executable instructions further cause the processor to rescale the reference grayscale value data to the reference interval.

In some examples, the grayscales in the image may first be sorted in an image histogram and then calculated as low and high percentiles, for example somewhere between the lowest 2-5% values and then rescaling to get rid of the highest and/or lowest 2-5% of these values. The image histogram may then be rescaled to the reference interval. In tests it is shown that getting rid of the highest and/or lowest 2-5% of the values may make the image segmentation algorithms perform better.

Alternatively this could also be performed by histogram matching. For a particular region of interest such as the brain you may already know specific grayscale histogram profiles that correspond to the distribution of tissues types within that region of interest. For example, in the brain for a given region of interest the distribution of bony tissue, grey matter, and white matter should have a well known distribution within a histogram. The reference gray scale values can be used to at least partially modify the magnetic resonance image so that its grey scales are modified to fit a histogram that functions with a particular segmentation model. Also the algorithm for performing the segmentation can be modified using the reference grey scale values.

In another embodiment, the meta data is contained in the DICOM header of the magnetic resonance image. This may be beneficial in that it may provide a new use for an existing DICOM header which enables the improvement of the segmentation algorithm.

In another aspect, the invention provides for a computer program product comprising machine-executable instructions for execution by a processor controlling the medical imaging system. Execution of the machine-executable instructions causes the processor to receive a magnetic resonance image. Execution of the machine-executable instructions further causes the processor to receive meta data descriptive of the magnetic resonance image. The meta data comprises reference grayscale value data for two or more tissue types. Execution of the machine-executable instructions further cause the processor to segment the magnetic resonance image using an image segmentation algorithm using the reference grayscale value data.

In another aspect, the invention provides for a method of operating a medical imaging system. The method comprises receiving a magnetic resonance image. The method further comprises receiving meta data descriptive of the magnetic resonance image. The meta data comprises reference grayscale value data for two or more tissue types. The method further comprises segmenting the magnetic resonance image using an image segmentation algorithm using the reference grayscale value data.

In another aspect, an example may provide for a medical imaging system that comprises a magnetic resonance imaging system. The medical imaging system may further comprise a memory containing machine-executable instructions and pulse sequence commands for controlling the magnetic resonance imaging system to acquire magnetic resonance data. The medical imaging system may further comprise a processor. Execution of the machine-executable instructions causes the processor to control the magnetic resonance imaging system to acquire the magnetic resonance data. Execution of the machine-executable instructions further causes the processor to reconstruct the magnetic resonance image from the magnetic resonance data. Execution of the machine-executable instructions further causes the processor to generate reference grayscale value data for two or more tissue types. The reference grayscale values are modeled at least partially using the pulse sequence commands and a magnetic resonance simulator.

Execution of the machine-executable instructions further causes the processor to generate meta data for the magnetic resonance image. The meta data comprises the reference grayscale value data.

In another aspect, examples provide for a medical imaging system comprising a magnetic resonance imaging system. The medical imaging system may further comprise a memory containing machine-executable instructions and pulse sequence commands for controlling the magnetic resonance imaging system to acquire magnetic resonance data. The medical imaging system may further comprise a processor for controlling the magnetic resonance imaging system. Execution of the machine-executable instructions may further cause the processor to control the magnetic resonance imaging system to acquire the magnetic resonance data. Execution of the machine-executable instructions may further cause the processor to reconstruct the magnetic resonance image from the magnetic resonance data. Execution of the machine-executable instructions may further cause the processor to identify multiple phantom regions corresponding to two or more tissue types within the magnetic resonance image. These multiple phantom regions may also be in a separate diagnostic or phantom magnetic resonance image in some examples. Execution of the machine-executable instructions further causes the processor to generate reference grayscale value data for the two or more tissue types. The reference grayscale value data is generated by calculating a statistical value of grayscale values within each of the multiple phantom regions. Execution of the machine-executable instructions further causes the processor to generate meta data for the magnetic resonance image. The meta data comprises the reference grayscale values.

It is understood that one or more of the aforementioned embodiments of the invention may be combined as long as the combined embodiments are not mutually exclusive.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as an apparatus, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer executable code embodied thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A ‘computer-readable storage medium’ as used herein encompasses any tangible storage medium which may store instructions which are executable by a processor of a computing device. The computer-readable storage medium may be referred to as a computer-readable non-transitory storage medium. The computer-readable storage medium may also be referred to as a tangible computer readable medium. In some embodiments, a computer-readable storage medium may also be able to store data which is able to be accessed by the processor of the computing device. Examples of computer-readable storage media include, but are not limited to: a floppy disk, a magnetic hard disk drive, a solid state hard disk, flash memory, a USB thumb drive, Random Access Memory (RAM), Read Only Memory (ROM), an optical disk, a magneto-optical disk, and the register file of the processor. Examples of optical disks include Compact Disks (CD) and Digital Versatile Disks (DVD), for example CD-ROM, CD-RW, CD-R, DVD-ROM, DVD-RW, or DVD-R disks. The term computer readable-storage medium also refers to various types of recording media capable of being accessed by the computer device via a network or communication link. For example a data may be retrieved over a modem, over the internet, or over a local area network. Computer executable code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with computer executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

‘Computer memory’ or ‘memory’ is an example of a computer-readable storage medium. Computer memory is any memory which is directly accessible to a processor. ‘Computer storage’ or ‘storage’ is a further example of a computer-readable storage medium. Computer storage may be any volatile or non-volatile computer-readable storage medium.

A ‘processor’ as used herein encompasses an electronic component which is able to execute a program or machine executable instruction or computer executable code. References to the computing device comprising “a processor” should be interpreted as possibly containing more than one processor or processing core. The processor may for instance be a multi-core processor. A processor may also refer to a collection of processors within a single computer system or distributed amongst multiple computer systems. The term computing device should also be interpreted to possibly refer to a collection or network of computing devices each comprising a processor or processors. The computer executable code may be executed by multiple processors that may be within the same computing device or which may even be distributed across multiple computing devices.

Computer executable code may comprise machine executable instructions or a program which causes a processor to perform an aspect of the present invention. Computer executable code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the C programming language or similar programming languages and compiled into machine executable instructions. In some instances the computer executable code may be in the form of a high level language or in a pre-compiled form and be used in conjunction with an interpreter which generates the machine executable instructions on the fly.

The computer executable code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Aspects of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It is understood that each block or a portion of the blocks of the flowchart, illustrations, and/or block diagrams, can be implemented by computer program instructions in form of computer executable code when applicable. It is further understood that, when not mutually exclusive, combinations of blocks in different flowcharts, illustrations, and/or block diagrams may be combined. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

A ‘user interface’ as used herein is an interface which allows a user or operator to interact with a computer or computer system. A ‘user interface’ may also be referred to as a ‘human interface device.’ A user interface may provide information or data to the operator and/or receive information or data from the operator. A user interface may enable input from an operator to be received by the computer and may provide output to the user from the computer. In other words, the user interface may allow an operator to control or manipulate a computer and the interface may allow the computer indicate the effects of the operator's control or manipulation. The display of data or information on a display or a graphical user interface is an example of providing information to an operator. The receiving of data through a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, webcam, headset, pedals, remote control, and accelerometer are all examples of user interface components which enable the receiving of information or data from an operator.

A ‘hardware interface’ as used herein encompasses an interface which enables the processor of a computer system to interact with and/or control an external computing device and/or apparatus. A hardware interface may allow a processor to send control signals or instructions to an external computing device and/or apparatus. A hardware interface may also enable a processor to exchange data with an external computing device and/or apparatus. Examples of a hardware interface include, but are not limited to: a universal serial bus, IEEE 1394 port, parallel port, IEEE 1284 port, serial port, RS-232 port, IEEE-488 port, bluetooth connection, wireless local area network connection, TCP/IP connection, ethernet connection, control voltage interface, MIDI interface, analog input interface, and digital input interface.

A ‘display’ or ‘display device’ as used herein encompasses an output device or a user interface adapted for displaying images or data. A display may output visual, audio, and or tactile data. Examples of a display include, but are not limited to: a computer monitor, a television screen, a touch screen, tactile electronic display, Braille screen,

Cathode ray tube (CRT), Storage tube, Bi-stable display, Electronic paper, Vector display, Flat panel display, Vacuum fluorescent display (VF), Light-emitting diode (LED) display, Electroluminescent display (ELD), Plasma display panel (PDP), Liquid crystal display (LCD), Organic light-emitting diode display (OLED), a projector, and Head-mounted display.

