TECHNICAL FIELD

The present invention relates to a method and system for automatically segmenting a blood vessel from medical image data. More specifically, the present invention relates to a method and system for automatically segmenting a blood vessel in a medical image by using machine learning and an image processing algorithm.

BACKGROUND ART

The three-dimensional medical image data includes information on various parts of the human body such as blood vessels and bones of a patient. In order to diagnose and treat a patient with a blood vessel abnormality, it is necessary to perform an operation of extracting blood vessels from three-dimensional medical image data and segmenting them. Currently, in the medical field, most of the operations of analyzing three-dimensional medical image data and extracting and segmenting blood vessels are performed manually. In the case of manually segmenting blood vessels, it takes a long time to segment the blood vessels, and errors in blood vessel segmentation may occur due to mistakes of an operator (doctor or technician). There is a problem that it is difficult to achieve standardization for obtaining the same result for each operator for the same three-dimensional medical image.

A technique that automatically detects blood vessels from three-dimensional medical image data without intervention of an operator, segments the blood vessels, and reconstructs the blood vessels into three-dimensional shape data can diagnose, treat and monitor diseases related to blood vessels more quickly and accurately.

For example, Non-Patent Document 1 discloses a technique (CT-FFR technique) in which a three-dimensional shape of a coronary artery is constructed from a patient's cardiac CT image, and hemodynamic simulation is performed on a three-dimensional coronary artery to non-invasively predict an FFR value and diagnose a coronary artery disease. If the coronary artery can be automatically detected from a cardiac CT image and can be segmented, it is possible to more quickly and accurately diagnose a disease caused by stenosis of the coronary artery through the use of the CT image. In addition, Patent Document 1 discloses a technique for constructing a three-dimensional shape of a cerebral blood vessel from medical images, and predicting the function of the cerebral blood vessel by performing fluid structure interaction modeling (FSI) and simulation. If the three-dimensional shape of the cerebral blood vessel can be automatically detected from the three-dimensional images of the brain and can be segmented, it is possible to quickly grasp the anatomical state of the cerebral blood vessel and utilize it for diagnosis. In addition, it is possible to predict the pressure and velocity of a blood flow in the cerebral blood vessel by performing rapid and accurate simulation for the cerebral blood vessel. Furthermore, it is possible to functionally grasp the state of the cerebral blood vessel, and use it for diagnosis.

Recently, a technology for segmenting a blood vessel through the use of machine learning has been developed. Patent Literature 2 discloses a technique for segmenting a blood vessel from a voxel through the use of a trained vessel orientation tensor classifier. In addition, techniques for segmenting blood vessels through the use of machine learning are disclosed in Non-Patent Documents 2 to 4. In addition, Patent Document 3 discloses a technique for automatically segmenting a blood vessel from three-dimensional medical image data through the intervention of a user.

As a result of the segmentation of a blood vessel attempted by the present inventors through the use of machine learning, it was found that there are a case where the portion which is not a blood vessel is segmented as a blood vessel (noise or outlier generation) and a case where the portion which is a blood vessel is excluded from the segmentation as if it is not a blood vessel (missing of a blood vessel region). As the cause of this error, it is presumed that the amount of data used for machine learning is insufficient, or incorrectly labeled mask data is used for machine learning. Eventually, it was found that if a blood vessel is segmented by a machine learning method at the current technical level, it is difficult to avoid some degree of blood vessel noise and missing of a blood vessel.

On the other hand, if a blood vessel is segmented from three-dimensional medical image data through the use of a known image processing algorithm, it is impossible to completely automate the segmentation of a blood vessel because the image processing algorithm known so far is not perfect. Particularly, in the segmentation of a coronary artery or the segmentation of a cerebral blood vessel, a method of completely and automatically segmenting the accurate shape and dimensions of a blood vessel to such a level as to enable hemodynamics simulation has not been developed. Therefore, it is necessary for an operator to input a seed before the start of segmentation, or it is necessary for an operator to remove or correct an erroneously segmented region during or after segmentation.

PRIOR ART DOCUMENT

Patent Document

1. US 2012/0203530 A1, METHOD AND SYSTEM FOR PATIENTS SPECIFIC COMPUTATIONAL MODELING AND SIMULATION FOR COUPLED HEMODYNAMIC ANALYSIS OF CEREBRAL VESSELS2. US 2017/0262981 A1, METHOD AND SYSTEM FOR MACHINE LEARNING BASED ESTIMATION OF ANISOTROPIC VESSEL ORIENTATION TENSOR3. U.S. Pat. No. 6,754,376 B1, METHOD FOR AUTOMATIC SEGMENTATION OF MEDICAL IMAGES

Non-Patent Document

1. Kwon, Soon-Sung, et al. “A novel patient-specific model to compute coronary fractional flow reserve.” Progress in biophysics and molecular biology 116.1 (2014): 48-55.2. Manoj, S., Sandeep PM Muralidharan, and M. Sandeep. “Neural network based classifier for retinal blood vessel segmentation.” International Journal of Recent Trends in Electrical & Electronics Engineering 3.1 (2013): 44-53.3. Moeskops, Pim, et al. “Deep learning for multi-task medical image segmentation in multiple modalities.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2016.4. Long, Jonathan, Evan Shelhamer, and Trevor Darrell. “Fully convolutional networks for semantic segmentation.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.5. Good fellow, Ian, et al. “Generative adversarial nets.” Advances in neural information processing systems. 2014.

SUMMARY

There is a need for a new method and system capable of fully automatically segmenting three-dimensional shape blood vessel data from three-dimensional medical image data in a rapid and accurate manner without intervention of an operator. An object of the present invention is to provide a novel blood vessel segmentation method and system that can meet such a need.

In particular, the present invention provides a new method and system capable of automatically segmenting a blood vessel without intervention of an operator by supplementing the deficiencies of a segmentation method using machine learning with a segmentation method using an image processing algorithm.

The method for segmenting three-dimensional blood vessel data from three-dimensional medical image data of a patient according to the present invention is executed in the form of a program in a computer system. The method for automatically segmenting three-dimensional blood vessel data according to the present invention includes: receiving a three-dimensional medical image data of a patient; generating three-dimensional shape machine-learning blood vessel data from the received three-dimensional medical image data through the use of a machine-learned segmentation program so as to generate three-dimensional blood vessel data; and generating corrected three-dimensional shape blood vessel data from the received three-dimensional medical image data and the generated three-dimensional shape machine-learning blood vessel data through the use of an image processing program. Furthermore, the three-dimensional machine-learning blood vessel data is composed of at least one noise data set other than a blood vessel region and a data set in which the blood vessel region is missing, and the image processing program is configured to compare the received three-dimensional medical image data and the generated three-dimensional shape machine-learning blood vessel data to match the blood vessel region, supplement the missing data set, and remove the noise data set, so as to generate corrected three-dimensional shape blood vessel data.

In the present invention, the three-dimensional medical image data may include CT images, MRI images and ultrasound images. The machine-learned segmentation program may include a convolutional neural network (CNN) algorithm and a fully convolutional network (FCN) algorithm.

In some embodiments, the machine-learned segmentation program may include a FCN algorithm. The FCN algorithm of the machine-learned segmentation program may be configured to convert axial slice images of three-dimensional medical images into bitmap images, label a blood vessel region of each of the converted bitmap images, convert each of the labeled bitmap images into a mask image to utilize the mask image as learning data, and perform learning by using each of the labeled bitmap images and the mask image as a pair.

In some embodiments, the machine-learned segmentation program may further include a GAN algorithm.

In some embodiments, the image processing program may be configured to supplement the missing data set through the use of a region growing algorithm for the blood vessel region of the received three-dimensional medical image data by using coordinate information of the generated three-dimensional shape blood vessel data as a seed, calculate a volume of the three-dimensional shape blood vessel data sets to which the missing data region is connected, determine blood vessel data whose calculated volume is equal to or less than a predetermined value as a noise data set, and remove the noise data set.

In some embodiments, the image processing program may be configured to segment a blood vessel region of the received three-dimensional medical image data by using coordinate information of the generated three-dimensional shape blood vessel data as a seed, calculate a volume of the segmented three-dimensional shape blood vessel data, determine blood vessel data whose calculated volume is equal to or less than a predetermined value as a noise data set, and remove the noise data set.

According to another aspect of the present invention, a medical image processing system for converting three-dimensional medical image data of a patient into three-dimensional blood vessel data through the use of a computer is provided. The system according to the present invention includes a computer installed with a computer program for performing the method recited above.

A method for automatically segmenting three-dimensional blood vessel data from three-dimensional medical image data of a patient through the use of a computer includes: receiving the three-dimensional medical image data of the patient; generating primary three-dimensional shape machine-learning blood vessel data by processing the received three-dimensional medical image data through the use of an artificial intelligence neural network; and generating secondary three-dimensional shape blood vessel data by processing the primary three-dimensional medical image data, wherein the act of generating secondary three-dimensional shape blood vessel data generates the secondary three-dimensional shape blood vessel data by using the primary three-dimensional shape machine-learning blood vessel data as a seed. In the present invention, the artificial intelligence neural network may include a CNN or a FCN.

In some embodiments, the act of generating the primary three-dimensional shape machine-learning blood vessel data through the use of the artificial intelligence neural network may include: constructing the artificial intelligence neural network; causing the artificial intelligence neural network to perform learning by processing three-dimensional medical images of a plurality of patients; and primarily segmenting the three-dimensional medical images of the patients, which is not used as a learning material in the learned artificial intelligence neural network. Furthermore, the act of constructing the artificial intelligence neural network includes: constructing a FCN for initial learning; using the FCN as a generator module to improve the neural network performance by applying a GAN algorithm; and constructing an inspector module to perform the GAN algorithm.

In the present invention, the three-dimensional shape blood vessel data primarily generated through the use of the artificial intelligence neural network may be composed of at least one noise data set other than a blood vessel region and at least one missing data set in which the blood vessel region is missing.

In some embodiments, the act of generating the secondary three-dimensional shape blood vessel data may include: matching the blood vessel region by comparing the primary three-dimensional shape blood vessel data with the received three-dimensional image data; connecting the blood vessel by supplement the missing data set; and removing the noise data set. Furthermore, the act of connecting the blood vessel by supplementing the missing data set may connect a missing data region through the use of a region growing algorithm for the blood vessel region of the received three-dimensional image data by using coordinate information of the generated primary three-dimensional shape blood vessel data as a seed. In addition, the act of removing the noise data set may include: calculating a volume of the three-dimensional blood vessel data sets to which the missing data region is connected; determining a blood vessel data set whose calculated volume is equal to or less than a predetermined value as noise data; and removing the blood vessel data set.

The method for segmenting three-dimensional blood vessel data from three-dimensional medical image data of a patient according to the present invention completely automatically segments the three-dimensional blood vessel data by supplementing the disadvantages of the machine learning and the image processing algorithm.

The method according to the present invention uses the three-dimensional shape blood vessel data generated by machine learning as a seed of an image processing algorithm. The three-dimensional shape blood vessel data generated by machine learning is that includes a noise data set having a segmented portion other than a blood vessel and does not include data for a blood vessel region. The three-dimensional image processing algorithm processes the original medical image data by using the three-dimensional shape blood vessel data generated by inaccurate machine learning as a seed. Thus, three-dimensional shape blood vessel data is generated by supplementing and connecting a missing blood vessel data set and removing a noise data set.

Accordingly, the method according to the present invention can completely automatically generate three-dimensional shape blood vessel data in a rapid and accurate manner without intervention of an operator.

