FIELD

This disclosure relates to image processing systems and methods to improve 3D segmentation and anatomy classification. In particular the disclosure is directed at improved techniques and method of identifying structures within computed tomography (CT) images and three-dimensional (3D) models derived therefrom, to improve surgical or treatment planning.

BACKGROUND

In many domains there's a need for segmenting and/or classifying voxels in a volumetric data. In terms of medical imaging, there are many open source and proprietary systems that enable manual segmentation and/or classification of medical images such as CT images. These systems typically require a clinician or a technician in support of a clinician to manually review the CT images and to effectively paint in the blood vessels or other structures manually, sometimes pixel by pixel. The user normally must scroll through many 2D slices and mark many pixels in order to obtain an accurate 3D segmentation/classification. As can be appreciated, such manual efforts are tedious and time-consuming rendering such methods very difficult to utilize for any type of surgical planning.

SUMMARY

One aspect of the disclosure is directed to a method of image processing including: acquiring a computed tomography (CT) image data set of the lungs; segmenting the CT image data set to identify airways and/or blood vessels in the CT image data set; skeletonizing the segmented CT image data by identifying the center points of the airways and/or blood vessels and forming a skeleton; graphing the skeleton to identify branches of the skeleton; assigning a branch identification (ID) to each branch of the skeleton; associate each voxel of the segmented CT image data set with a branch ID, where the branch ID of the voxel is the same as the branch ID of the closest center point. The method of image processing also includes generating a three-dimensional (3D) mesh model from the graph of the skeleton. The method of image processing also includes associating each vertex of the 3D mesh model with a branch id, and displaying in a user interface the 3D mesh model and a slice image from the image data set, where portions of the 3D mesh model that appear in slice image are highlighted. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods and systems described herein.

A further aspect of the disclosure is directed to a method of image processing including: segmenting an image data set, skeletonizing the segmented image data set, graphing the skeletonized image data set, assigning a branch identification (ID) for each branch in the graph, associate each voxel of the segmented image data set with a branch id. The method of image processing also includes generating a three-dimensional (3D) mesh model from the graphed skeletonized image data set; and associating each vertex of the 3D mesh model with a branch ID. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods and systems described herein.

Implementations of this aspect of the disclosure may include one or more of the following features. The method where a plurality of vertices with the same branch ID form an object. The method where each pixel of the segmented image data set is associated with an object in the 3D mesh model based on their branch ID. The method further including presenting the 3D mesh model and the image data set on a user interface. The method where the image data set is presented on the user interface as a slice image of the image data set. The method where any portion of the slice image that corresponds to a portion of an object of the 3D mesh model is colored the same as the corresponding object. The method where the color of an object in the 3D mesh model may be changed upon receipt of an appropriate command. The method where change of color of the object results in a change in color of a corresponding portion of the image. The method where the slice images of the image data set are scrollable. The method further including receiving a selection of a pixel in the displayed image of the image data set. The method may also include determining if a branch ID is associated with the selected pixel. The method may also include highlighting all pixels in the displayed image having the same branch ID in a common color when the selected pixel is associated with a branch id. The method further including highlighting in the user interface an object of the 3D mesh model having the same branch ID as the selected pixel in the image data set. The method further including receiving a selection of an object in a displayed 3D mesh model; determining the branch ID of the object. The method may also include displaying all pixels in a displayed image of the image data set having the same branch ID as the selected branch in a common color. The method further including highlighting the object in the 3D mesh model in a contrasting color. The method further including defining a cluster ID for each branch. The method further including displaying all objects having a common cluster ID in a common color. The method where the cluster ID is based on a commonality of the objects of the cluster. The method where the commonality of the objects of the cluster is based on selecting the objects by the smallest angle of intersection between connected branches and objects. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium, including software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

One aspect of the disclosure is directed to a system including: a processor in communication with a display, and a computer readable recording medium storing thereon instructions that when executed by the processor: read an image data set from the computer readable recording medium, segment the image data set, skeletonize the segmented image data set, graph the skeletonized image data set, assign a branch identification (ID) for each branch in the graph, associate each voxel of the segmented image data set with a branch ID, generate a three-dimensional (3D) mesh model from the graphed skeletonized image data set. The system also causes the processor to associate each vertex of the 3D mesh model with a branch ID. The system also includes display in a user interface the 3D mesh model and a slice image from the image data set, where portions of the 3D mesh model that appear in slice image are highlighted. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods and systems described herein.

Implementations of this aspect of the disclosure may include one or more of the following features. The system where the instructions that when executed by the processor: receive a selection of a pixel in the displayed slice image, determine if a branch ID is associated with the selected pixel, and highlight all pixels in the displayed image having the same branch ID in a common color when the selected pixel is associated with a branch ID, or receive a selection of an object in a displayed 3D mesh model, determine the branch ID of the object, and display all pixels in the slice image having the same branch ID as the selected branch in a common color. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium, including software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

