Hierarchical Deep Network with Uncertainty-aware Semi-supervised Learning for Vessel Segmentation

The analysis of organ vessels is essential for computer-aided diagnosis and surgical planning. But it is not a easy task since the fine-detailed connected regions of organ vessel bring a lot of ambiguity in vessel segmentation and sub-type recognition, especially for the low-contrast capillary regions. Furthermore, recent two-staged approaches would accumulate and even amplify these inaccuracies from the first-stage whole vessel segmentation into the second-stage sub-type vessel pixel-wise classification. Moreover, the scarcity of manual annotation in organ vessels poses another challenge. In this paper, to address the above issues, we propose a hierarchical deep network where an attention mechanism localizes the low-contrast capillary regions guided by the whole vessels, and enhance the spatial activation in those areas for the sub-type vessels. In addition, we propose an uncertainty-aware semi-supervised training framework to alleviate the annotation-hungry limitation of deep models. The proposed method achieves the state-of-the-art performance in the benchmarks of both retinal artery/vein segmentation in fundus images and liver portal/hepatic vessel segmentation in CT images.


Introduction
The analysis and visualization of vessels in human organs are essential for diagnosis and treatment of various diseases. For example, automatic retinal artery and vein (A/V) segmentation assists doctors diagnose systemic and cardiovascular diseases [1,22]. Specifically, the asymmetrical change of retinal A/V measured by arteriolar-venular ratio (AVR), is closely associated with several cardiovascular diseases [20,51]. Decrease in AVR is a biomarker with increased risk of stroke [22]. Meanwhile, the narrowing of retinal arteriolar caliber has been reported to be related to the risk of hypertension and diabetes development [4,36]. Similarly, accurate segmentation and visualization of liver vessels are key prerequisites for safe and efficient surgery in the liver regions [6,44]. The analysis of hepatic and portal vessels can also provide valuable diagnostic information with regards to chronic liver disease and cirrhosis [39]. In a word, these clinical requirements are summarized as the accurate segmentation of organ vessels as well as their sub-types, e.g., the retinal vessels with arteries and veins in fundus images, and the liver vessels with hepatic and portal sub-types in liver CTs.
Although this field has received considerable research attention, several limitations hinder the routine application of vessel segmentation and sub-type classification in the clinical practice. First, the low contrast between capillaries and background and between different subtype vessels poses a challenge in segmentation models. An example of retinal fundus image is shown in Figure  1. Some capillary vessels in the areas far away from an optic disk are too indiscernible to be detect and the contrast of A/V is very low there. The situation is similar when observing portal and hepatic vessels in liver anatomical CT images, as shown in Figure 1. To address this challenge, we think the model should focus more on those low-contrast areas, which are usually capillaries, vessel edges, etc. We discover that the unconfident predictions of whole vessel segmentation are often generated from the low-contrast regions, as shown in Figure 6. With this location information as a guide, we can spatially activate the feature representation of sub-type vessel segmentation, which enhances the spatial attention of the model on those 'hard' regions for the task of sub-type vessel segmentation.
Besides the above limitation, the previous methods that segment vessels in a two-stage regime, which first segment the whole vessels, then classify the results into sub-types, may propagate and even amplify the firststage errors mainly caused by the low-contrast regions to the subsequent stage. Instead, the multi-task fashion has attracted long-standing attention and proved to be effective in other medical practices [49,3]. For example, the task of liver lesion segmentation is usually coupled with the liver segmentation in a multi-task manner. Similar in spirit, we integrate the segmentation of whole and sub-type vessels as a one-stage multi-task framework, to improve the feature sharing between the two tasks, alleviate the accumulated errors and further overcome the challenge caused by low-contrast regions.
Another challenge is the scarcity of the pixel-wise annotation for vessels and their sub-types. Usually, the labeling process for pixel-wise vessels requires radiology expertise and is laborious as well as time-consuming, which causes the acquiring labels for every image in a training dataset to be a burden. Thus, a more practical situation in practical clinical applications is that only some training data are labeled. To address this, we leverage the information from massive unlabeled datasets via semi-supervised learning methods. A common approach is self-labeling, where the model is trained with labeled data by supervised methods and then allocates pseudo labels for unannotated data. However, the pseudo-generated labels are not always reliable due to the distribution shift between labeled and unlabeled data, which initiates the requirement of filtering unreliable pseudo labels. Our intuition is that if an unlabeled sample is allocated a more consistent pseudo label, i.e., maintains the same class, we think its pseudo label is high-quality and reliable. Thus, we introduce the uncertainty estimation for the generated labels, and retain only the ones with enough certainty for the subsequent optimization. Overall, in this paper, we propose a hierarchical deep network with uncertainty-aware semi-supervised learning for organ vessel segmentation. First of all, we propose the hierarchical capillary-enhanced network for joint whole and sub-type vessel segmentation, where the multi-task design completes both sub-tasks, emphasizes on the capillary-like low-contrast regions based on the guide information from the sub-task of whole vessels segmentation, and regularizes the model to avoid the error propagation occurring in prior two-stage works. Moreover, we develop a uncertainty-aware semi-supervised learning method to efficiently leverage information from massive unlabeled datasets. We allow the model to selflabel the unlabeled data and filter the low-quality pseudo labels. To further model the reliability of the generated labels, we introduce the uncertainty-aware estimation and filter the pseudo-labeled data with high uncertainty.
Our initial work has been published on MICCAI 2019 [32]. We extend the conference version by incorporating a novel semi-supervised learning framework by uncertainty modeling. We also enrich the experimental section by validating the proposed model on liver vessel segmentation.

Related work
In this section, the previous models for organ vessel and sub-type vessel segmentation are reviewed. The semisupervised learning methods that have been employed in the case of shortage of labeled data are then reviewed.

Organ Vessel Segmentation
In graph-based approaches, a vascular graph is built from the extracted vessel centerlines and then each individual graph tree is classified into arteries and veins. The model proposed by Niemeijer et al. [38] extracts handcrafted features from the vessel centerlines and then performs classification based on soft labels. Dashtbozorg et al. [7] designed a model for retinal A/V segmentation through a pipeline of whole vessel segmentation, centerline extraction and graph reconstruction. Estrada et al. [11] proposed a graph-theoretic framework for A/V segmentation by estimating the vascular topology using a global likelihood model and domain specific features. Xu et al. [53] extracted novel texture features from vessel centerlines and employed KNN as the classifier. In the recent work from Zhao et al. [57], the dominant set theory was used to identify the complicated branches and crossover junctions, building a vascular graph and classifying the centerlines into artery and vein. In terms of liver vessel segmentation, a contextual information extraction was used in a hierarchical voting mechanism in [5] to segment liver vessel. Incorporating knowledge of liver vascular structures, Jerman et al. [21] employed vascular filters based on Hessian filters with a loss constraint ratio to the vessel shape. Merveille et al. [33] utilized morphological filters by ranking the orientation responses of path operators. Lebre et al. [26] further proposed a 'Couinaud' scheme for the portal/hepatic vessel segmentation. Guo et al. [15] modeled a vascular network based on graph cut, thinning and vessel combination. Nevertheless, the above two-step methods suffer from the limitation that the performance of sub-type vessel classification affects the accuracy of the whole vessel segmentation, especially for the graph-based methods. Additionally, feature-engineering methods heavily rely on the fine design of hand-crafted feature extraction, which presents difficulty in capturing complex vessel patterns.
With the rapid development of deep neural networks, the number of studies using Fully Convolutional Networks (FCN) to detect and classify organ vessels is increasing. AlBadawi and Fraz [2] adopted the FCN with an encoder-decoder structure for the pixel-wise prediction of retinal arteries and veins. Meyer et al. [34] also used FCN for the same task and reported the performance on major vessels thicker than three pixels. Huang et al. [18] utilized FCN 3D-UNet for the vessel segmentation in liver CT images. Deep learning based methods have demonstrated their potential to segment sub-type vessels in an end-to-end manner. However, these models applied direct segmentation without exploiting contextual image information, thus showing uncertainty in prediction and often classifying pixels on sub-type vessels especially in capillary as background. Thus there is still room for improvements if the specific model design could be proposed to account for the ambiguity in prediction and incorporating proper contextual information.

Semi-supervised Learning
With the aim at exploiting a large number of unlabeled data, plenty of researches raised the focus on semisupervised learning strategy [16,19,29,50,41]. Hong et al. [16] proposed decoupled networks including classification and segmentation networks, where bridging layers can deliver class-specific information. Hung et al. [19] utilized an adversarial learning mechanism and adopted a logit (probability maps) output from network as a confidence map in semi-supervised learning. Moreover, some approaches also designed unsupervised loss into the overall loss function for semi-supervised learning [55,25,48]. Sajjadi et al. [43] proposed a consistency loss on the outputs of the model conditioned on random perturbations of one image to impose transform invariance. In some prior literature, researchers assigned pseudo-labels to the massive unlabeled datasets [27,45]. Lee et al. [27] utilized a pre-trained network to generate pseudo-labels of unlabeled data by itself, which were then used for finetuning the model to achieve better robustness.

Method
A data pre-processing pipeline and a hierachical network are described in this section. Then the design of spatial activation to enhance low responses in subtype vessels in capillary regions is detailed. Finally, the uncertainty-based semi-supervised framework built on  a probabilistic model is presented. Please note that a retinal fundus image is used to illustrate our approach.

Data Pre-processing
We use specific pre-processing on different organs and modalities. For retinal fundus images, we apply a series of pre-processing on raw images shown in the green box in Figure 2. To correct the bias generated by nonuniform illumination, a illumination correction is applied. Then a line detector [37] and Gabor filter [46] are utilized on corrected images in parallel for edge extraction. Raw images, illumination-corrected images and two edge maps are further cropped into pairs of patches and are further concatenated in the channel dimension to form an 8-channel input (i.e., 3 channels for a raw/corrected image, 1 channel for either of the two kinds of edge map). In the ablation study, we validate the effectiveness of this proposed multiple-inputs design. In the testing phase, a sliding window strategy is applied on the test image for the patch-wise predictions, as shown in Figure 2, then the patches are further merged to reconstruct the image prediction via averaging on logits of overlapping patches as post-processing.
For liver vessel segmentation validated on 3D liver CT scans, we randomly crop volumetric cubes from normalized liver CT scans as model input without other pre-processing operations in the training phase. A similar sliding-window approach is applied in testing phase.

Hierachical Capillary-enhanced Network
Here we present the architecture of our hierarchical capillary-enhanced network in Figure 3 for joint whole and sub-type vessel segmentation. A U-Net like architecture is adopted as backbone, since U-Net [42] has proven its potency in segmenting medical images accurately and efficiently. Similar to the regular U-Net, we concatenate the feature maps from the previous decoding layer with the corresponding feature maps from the encoding layers to be used as input in each decoding layer.
To predict the whole vessel and sub-type vessels (i.e., arteries/veins in retinal image and portal/hepatic vessels in liver image) simultaneously, we design an adhoc output block where contextual information from whole vessel segmentation helps guide the sub-type vessel segmentation, as shown in Figure 3 (A) . The motivation of this design is building a hierarchical model to aid a difficult task (sub-structure segmentation) with information from an easy task (whole vessel segmentation) and avoid accumulative segmentation error.
We apply two different sets of convolution on the input features of the output block to generate two chunks of feature maps. For whole vessel segmentation, one chunk of features is squeezed into a 1-channel representation via 1 × 1 convolution + BN, then a Sigmoid function is utilized to produce a logit ranging from 0 to 1, where a binary cross-entropy loss function L v is imposed. Therefore, the output block is able to predict whole vessel segmentation while the resulting logit describes the contextual information of the confidence level of the model in making a prediction about a certain pixel belonging to a vessel instead of background. In this stream the common characteristics of sub-type vessels are extracted as the whole vessel regions to be distinguished from backgrounds.
As shown in the output block in Figure 3 (A), two chunks of features obtained by convolving input features are concatenated together in another stream and weighted by the spatial activation which is discussed in the next subsection. Then the weighted features are further forwarded into a 1 × 1 convolution + BN + Sigmoid layer, squeezing them into a logit of N channels with N denoting total number of sub-type vessel classes. Each channel in the logit represents the likelihood of this sub-type class being presented in the prediction. In this paper, N equals 2 for both retinal and liver vessel segmentation. Then each channel of the logit is used for computing the binary cross-entropy loss to train the segmentation of sub-type vessels L si where subscription i represents a certain sub-type vessel. Note that the segmentation of sub-type vessels is not a one-hot problem since a pixel is likely to be classified into both artery and vein if it lies in the junction of vessels. Thus either channel in the two-channel logits represents the possibility of one substructure of vessel. In this stream, the model learns to differentiate sub-type vessels (artery/vein in the retinal image or portal/hepatic vessel in the liver image). To designing such a two-stream multi-task output block, we exploit both the homogenous and heterogenous characteristics of all sub-type vessels.

Spatial Activation
Here a detailed motivation and description of the spatial activation of our capillary-enhanced network is presented. From Figure 1, we observe the low-contrast intensity of capillary vessels compared with the background, which leads to uncertainty from the segmentation model in classifying them. To validate this hypothesis, we extract the probability heatmap of predicting the whole vessel and visualize it in Figure 4. In Figure 4 (A), the golden standard vessel segmentation is shown. In Figure 4   The attention maps is obtained by the activation function m(x) referred in Figure 5.
ever, the pixels in the areas of capillary vessels range between (0.3,0.7], indicating that the network is highly uncertain in predicting these capillary vessels, decreasing the network performance in sensitivity.
To further utilize the contextual information of vessel segmentation to promote the difficult sub-type vessel segmentation, an ad-hoc activation function is proposed, as in Equation 1: where σ denotes the activation factor and is set as the identity mapping in our design, x denotes the value of each pixel on the probability map with regards to whole vessel segmentation. In Figure 5 we show the graph of this activation function. It can be seen that as the value of x approaches 0.5, the maximum 1 + σ(1 − e − 1 4 ) of the function is approximated. As the value of x approaches 0 or 1, then m(x) collapses to the identity mapping. We can pass the logit (probability map) of the whole vessel segmentation to this activation function and receive an attention weighting map. We then apply the attention score map to weight each feature maps from sub-type segmentation via point-wise multiplication.
In such a design, capillary vessels with pixel value around 0.5 will be enhanced with higher weights close to 1 + σ(1 − e − 1 4 ). Meanwhile, background or thick vessels that are easier to segment will remain unchanged with weights close to 1. An example of the activation map is presented in Figure 6.
A similar weighting approach is the focal loss [30] where hard examples are assigned larger weights automatically. In our case the hard example is the accurate segmentation of the capillary vessels. However, what distinguishes our work from the focal loss is we regularize the hard examples based on another related but easier task, i.e., the vessel segmentation in an hierarchical fashion. Additionally, focal loss is only used during training while our hierarchical spatial activation is applied on both training and testing phase.

Deep Supervision
Lacking sufficient amount of labeled images often leads to over-fitting of a model. To deal with this problem, we adopt a deep supervision [10] method, as shown in Figure 3 (B). By adding extra branches, the shallow layers can be guided closer to supervision information, which enables low-level layers to extract more semantics [56].
We design the loss function including binary crossentropy loss of the main branch, deep supervision branches and a weight decay regularization term, as in Equation 2 and 3: where Θ denotes the network parameters; pred o denotes the output of the segmentation network; i denotes the i th deep supervision block; c denotes the c th class; the weight of each class is denoted as µ c with 3 7 , 2 7 and 2 7 for vessel, artery and vein in retinal image and portal/hepatic vessel segmentation. The binary crossentropy is applied on every pixel in cropped patches. Fig. 7: The overall architecture of the proposed uncertainty-aware semi-supervised learning framework. The sampling process is designed to generate the pseudo annotation as well as its uncertainty estimation for an uncertainty-aware self-labeling program. The training process allows the model to capture the semantic-correlated pattern for the input unlabeled image. These two processes are alternative as well as progressive and can completes each other. Besides the proposed hierarchical segmentation network, we introduce a prior network and posterior network to implement the above processes.

Uncertainty-aware Semi-supervised Segmentation
Labeling vessel images is laborious and requires expert knowledge. To further leverage massive unlabeled data for better generalization, we propose a semi-supervised learning framework. Generating pseudo labels for the unlabeled data by an automatic annotation algorithm is a popular approach in semi-supervised learning. However, simply applying a deterministic model on the unlabeled dataset for pseudo-label acquisition ignores the common distribution shift between labeled data and unlabeled data. Once trained, the model based on labeled data may generate low-quality pseudo-labels for unlabeled data. Another major drawback of this labeling strategy lies in the fact that the annotation is performed by hard thresholding on the logits without handling the uncertainty in making such predictions, which leads to bias in pseudo-labels. To tackle these problems, we improve the pseudo-label generation by introducing the scheme of uncertainty estimation, which has been investigated in medical imaging practice [9,28,54,14]. Inspired by a probabilistic U-Net for segmentation [24], we propose an uncertainty-aware architecture based on Bayesian networks [23], which is shown to be more efficient than Monte Carlo dropout [8].
The overview of the training and sampling process of our proposed uncertainty-aware semi-supervised segmentation method is shown in Figure 7. Besides the proposed hierarchical segmentation network, we further introduce the prior network and the posterior network, to capture the prior and posterior latent representations of the input image, respectively. At a high level, the prior network is trained to embed similar representations to those by the posterior network with a Kullback-Leibler (KL) loss. Then in the sampling process, the latent representations captured by the trained prior network are expected to be close to the posterior representations and carry the label information. Following the spirit of conditional variational autoencoder (CVAE), the latent space is modeled as an N-dimensional axisaligned Gaussian distribution with parameters as mean µ and covariance σ 2 .
More precisely, in the training process, we train the prior network with the assist of a posterior network. The posterior network is fed with the input image and its ground truth and it is trained to generate the distribution that carries the information of semantic labels, which we call as posterior distribution. Then we utilize a Kullback-Leibler (KL) divergence to constrain the prior distribution generated from the prior network to approach the posterior distribution. Once trained, the prior network is regarded to be able to capture a similar pattern to the posterior network, and the prior distribution is approximate to the posterior distribution.
Then in the sampling process, we can draw a sample z i from the approximate posterior distribution from the trained prior network. We broadcast it to an N-channel feature map and concatenate it with the output of the hierarchical segmentation network. By repeating the distribution sampling M times, M corresponding model outputs can be obtained. Then the pixel-wise mean values and variance values among them are calculated as the pseudo annotation and the uncertainty estimation. Please note that for an input image, the output of the segmentor (hierarchical capillary-enhanced network) is invariant during different samples of latent space, so the computation of these forward passes is cheap.
With the pseudo-labels with uncertainty estimation, we can improve the hierarchical capillary-enhanced model by retraining only the pseudo-labels with high confidence. Assume that the model uncertainty of input image is U , which is expressed as the variance computed by several forward passes of the sampling. The notation u v represents the uncertainty at the v-th pixel in the pseudo-label. The Loss function of the retraining can be formulated in Equation 4 as: where I(·) is the indicator function outputting 1 if the condition is satisfied and 0 otherwise; pred and pseudo are the segmentation prediction and the pseudo label at the v-th pixel; H is a threshold to filter out uncertain label empirically set as 0.7. With this uncertainty-aware loss function, the segmentation network can learn from the reliable pseudo labels as well as avoid the possible misleading from unreliable ones. The overall algorithm for the proposed semi-supervised learning framework is list in Algorithm 1.

Algorithm 1 Overall flow of the proposed semisupervised learning framework
Input: labeled dataset D l , unlabeled dataset D u ; pretraining iterations T pre , retraining iterations T re ; initialized parameter of segmentation network θ, initialized parameter of prior and posterior networks θ . Output: trained parameter θ of segmentation network. 1: for loop = 1 to T pre do 2: Optimize θ and θ on labeled dataset D l by the training process in Figure 7. Take multiple forward passes of pretrained model to generate outputs on unlabeled dataset D u by the sampling process in Figure 7.

6:
Calculate the mean values of the multiple outputs as pseudo annotation.

7:
Calculate the variance values of the multiple outputs as uncertainty estimation.

8:
Optimize θ on dataset D u with the pseudo annotation and uncertainty estimation using Equation 4; 9: end for 4 Experimental Results

Dataset
For retinal artery/vein segmentation, we trained and evaluated our model on the publicly available AV-DRIVE database [17]. The AV-DRIVE database contains 20 training and 20 testing colorful retinal fundus images with dimension of 584 × 565 pixels, with pixel-wise labeling of vessel segmentation and A/V classification provided. To validate our semi-supervised learning approach, we pretrained segmentation model on AV-DRIVE dataset and generate pseudo annotations on HRF [40]  For the liver portal/hepatic vessel segmentation, we adopted the IRCAD datasets containing 3D CT-scans of 20 different imaging subjects. All the 20 scans are labeled with the pixels of portal and hepatic vessel. We conducted K-fold cross-evaluation and report the mean results of them following prior study [26,33,18,21,15].

Implementation Details
In the training stage, the stochastic gradient descent algorithm with momentum was adopted to optimize our model. The initial learning rate was set as 0.05 and halved every 5000 iterations. The size of the training mini-batch was 16. Multi-scale patches with size of 64 × 64, 96 × 96, 128 × 128 were randomly cropped from the retinal images during training and were further resized to 64 × 64 to exploit multi-scale information. In the testing stage, patches with size of 64 × 64 were extracted at the stride of 10. The segmentation of sub-type vessels and full vessel of a image were obtained by aggregating the patches, with the overlapping regions being averaged as post-processing. The experiments were conducted on a single NVIDIA GPU (Tesla P40) and the training process took about 3 hours.

Evaluation Metrics
We adopt common evaluation metrics: Accuracy(Acc), Sensitivity(Sen), Specificity(Sp), Area Under Curve(AUC). The Accuracy, Sensitivity and Specificity are defined as The metric AUC is defined as the receiver operating characteristics (ROC) area under the curve, which represents the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one.

Ablation Studies
On the retinal fundus image dataset AV-DRIVE, three groups of ablation studies are performed to evaluate the contribution of each design in our approach: 1) Joint prediction of vessel segmentation and artery/vein segmentation in a multi-task manner (denoted as MTs); 2) Multiple inputs using pre-processed images (denoted as MIs); 3) Spatial activation (denoted as AC). Baseline is built by removing the MTs, MIs and AC designs.
As indicated in Table 1, a multi-task way of performing vessel segmentation and sub-type segmentation simultaneously can improve both tasks. Also, by inputting multiple pre-processed images into the model, the A/V classification accuracy is improved by 0.3%. When the spatial activation is applied, the A/V segmentation can be further increased by 0.6%. Our proposed hierarchical capillary-enhanced network achieves a pixel-wise accuracy of 95.70% for vessel segmentation and 92.58% for sub-type vessel segmentation. We also compare our spatial activation with the design of focal loss. We observe no significant improvement when removing the activation block and equipping the crossentropy loss with focal loss. In Figure 9, an example is shown to compare our model with baseline. We zoom in multiple local areas, and our proposed method has remarkably improved the segmentation result of retinal artery and vein over the baseline by the MTs, MIs and AC design.
To evaluate our semi-supervised learning framework, we pretrain a segmentation model using the labeled     The comparative experimental results on A/V segmentation can be seen in Table 2. The Co-training model achieves better segmentation than baseline Supervised model, due to the augmented data size. Besides, when applying our uncertainty estimation model, semi-supervised method produces more accurate segmentation than the one using MC Dropout, approaching the upper bound Co-training. The vanilla annotation in such a semisupervised framework brings little improvements, because it introduces massive unreliable pseudo labels where the model is misled.

Comparison to Existing Methods
We compare our hierarchical capillary-enhanced network with other state-of-the-art approaches in the performances of whole retinal fundus vessel and artery/vein segmentation, as shown in Table 3 and Table 4.  In Table 3, we compare the capillary-enhanced network with other state-of-the-art models for whole retinal vessel segmentation on the testing dataset in AV-DRIVE. Our approach achieves the better performance among the compared models, owing to the joint optimization for both whole and sub-type vessel segmentation.
In Table 4, we compare our capillary-enhanced network with other state-of-the-art models for retinal arteries/veins segmentation on the testing dataset in both AV-DRIVE and INSPIRE datasets. When performing under a fair comparison, the proposed model achieves a pixel-wise accuracy of 94.50%, which outperforms the current state-of-the-art A/V segmentation method by a considerable margin.
We also have validated our algorithm on liver vessel segmentation in IRCAD [47]. Because 3D liver Computed Tomography (CT) images are provided in this dataset, we adopt a 3D U-Net, incorporating the proposed designed spatial activation and removing our multiinput module. In this database, we use our model to segment the portal and hepatic vessels in a multi-task manner. Then we combine them to form the segmentation of whole veins in a liver. The 3D reconstructions of the segmented portal and hepatic vessels are shown in Figure 10. We have compared our model with extensive prior approaches, as shown in Table 5. Note that we only report performances on whole vessel segmentation of some compared methods [18,21,15], because they were proposed for targeting on whole vessel regions. At last, the proposed method achieves superior performance against other state-of-the-art meth-ods. The common improvements in the joint task and the two sub-tasks also demonstrate the extensive effectiveness of the proposed multi-task mechanism

Conclusion
In this paper, we build the learning model that faces several serious practical challenges in organ vessel segmentation. First of all, we propose the hierarchical capillaryenhanced network to enhance the performance of vessel segmentation, especially for capillary regions. The proposed method enables the end-to-end segmentation of whole vessels and their sub-types simultaneously in a multi-task manner and allows the sub-tasks can benefit from the joint task. Moreover, we develop an uncertaintyaware semi-supervised learning framework to alleviate the practical issue of annotation scarcity. We evaluate the proposed overall method on some publicly available databases and compare it to the prior methods extensively in both the retinal fundus datasets and liver CT set. The experimental results show that our hierarchical capillary-enhanced model outperforms the stateof-the-art supervised methods under the fully annotated setting and other semi-supervised methods under the annotation-scarcity setting, which demonstrates the proposed model is effective extensively in the medical practice.
To summary, the proposed method significantly improves the accuracy and efficiency of A/V segmentation as well as alleviates the rely upon vessel annotation, which are both key requirements in real-world clinical applications. In the future work, we will go on working based on this research and develop a fully automatic and annotation-efficient machine for vascular parameter modeling to facilitate the clinical vascular biomarker study.