Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> Retinal vessel segmentation is a fundamental step for various ocular imaging applications. In this paper, we formulate the retinal vessel segmentation problem as a boundary detection task and solve it using a novel deep learning architecture. Our method is based on two key ideas: (1) applying a multi-scale and multi-level Convolutional Neural Network (CNN) with a side-output layer to learn a rich hierarchical representation, and (2) utilizing a Conditional Random Field (CRF) to model the long-range interactions between pixels. We combine the CNN and CRF layers into an integrated deep network called DeepVessel. Our experiments show that the DeepVessel system achieves state-of-the-art retinal vessel segmentation performance on the DRIVE, STARE, and CHASE_DB1 datasets with an efficient running time. <s> BIB001 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> The condition of the vascular network of human eye is an important diagnostic factor in ophthalmology. Its segmentation in fundus imaging is a nontrivial task due to variable size of vessels, relatively low contrast, and potential presence of pathologies like microaneurysms and hemorrhages. Many algorithms, both unsupervised and supervised, have been proposed for this purpose in the past. We propose a supervised segmentation technique that uses a deep neural network trained on a large (up to 400 $\thinspace$ 000) sample of examples preprocessed with global contrast normalization, zero-phase whitening, and augmented using geometric transformations and gamma corrections. Several variants of the method are considered, including structured prediction, where a network classifies multiple pixels simultaneously. When applied to standard benchmarks of fundus imaging, the DRIVE, STARE, and CHASE databases, the networks significantly outperform the previous algorithms on the area under ROC curve measure (up to $ > {0.99}$ ) and accuracy of classification (up to $ > {0.97}$ ). The method is also resistant to the phenomenon of central vessel reflex, sensitive in detection of fine vessels ( ${\rm sensitivity} > {0.87}$ ), and fares well on pathological cases. <s> BIB002 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> We have developed and trained a convolutional neural network to automatically and simultaneously segment optic disc, fovea and blood vessels. Fundus images were normalized before segmentation was performed to enforce consistency in background lighting and contrast. For every effective point in the fundus image, our algorithm extracted three channels of input from the point’s neighbourhood and forwarded the response across the 7-layer network. The output layer consists of four neurons, representing background, optic disc, fovea and blood vessels. In average, our segmentation correctly classified 92.68% of the ground truths (on the testing set from Drive database). The highest accuracy achieved on a single image was 94.54%, the lowest 88.85%. A single convolutional neural network can be used not just to segment blood vessels, but also optic disc and fovea with good accuracy. <s> BIB003 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> Accurate vessel segmentation is a tough task for various medical images applications especially the segmentation of retinal images vessels. A computerised algorithm is required for analysing the progress of eye diseases. A variety of computerised retinal segmentation methods have been proposed but almost all methods to date show low sensitivity for narrowly low contrast vessels. We propose a new retinal vessel segmentation algorithm to address the issue of low sensitivity. The proposed method introduces a deep learning model along with pre-processing and post-processing. The pre-processing is used to handle the issue of uneven illuminations. We design a fully Convolutional Neural Network (CNN) and train it to get fine vessels observation.The post-processing step is used to remove the background noise pixels to achieve well-segmented vessels. The proposed segmentation method gives good segmented images especially for detecting tiny vessels. We evaluate our method on the commonly used publicly available databases: DRIVE and STARE databases. The higher sensitivity of 75% leads to proper detection of tiny vessels with an accuracy of 94.7%. <s> BIB004 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> Retinal vessel information is helpful in retinal disease screening and diagnosis. Retinal vessel segmentation provides useful information about vessels and can be used by physicians during intraocular surgery and retinal diagnostic operations. Convolutional neural networks (CNNs) are powerful tools for classification and segmentation of medical images. Complexity of CNNs makes it difficult to implement them in portable devices such as binocular indirect ophthalmoscopes. In this paper a simplification approach is proposed for CNNs based on combination of quantization and pruning. Fully connected layers are quantized and convolutional layers are pruned to have a simple and efficient network structure. Experiments on images of the STARE dataset show that our simplified network is able to segment retinal vessels with acceptable accuracy and low complexity. <s> BIB005 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> Fundus images are one of the main methods for diagnosing eye diseases in modern medicine. The vascular segmentation of fundus images is an essential step in quantitative disease analysis. Based on the previous studies, we found that the category imbalance is one of the main reasons that restrict the improvement of segmentation accuracy. This paper presents a new method for supervised retinal vessel segmentation that can effectively solve the above problems. In recent years, it is a popular method that using deep learning to solve retinal vessel segmentation. We have improved the loss function for deep learning in order to better handle category imbalances. By using a multi-scale convolutional neural network structure and label processing approach, our results have reached the most advanced level. Our approach is a meaningful attempt to improve blood vessel segmentation and further improve the diagnostic level of eye diseases. <s> BIB006 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> Abstract Background and objective The condition of vessel of the human eye is an important factor for the diagnosis of ophthalmological diseases. Vessel segmentation in fundus images is a challenging task due to complex vessel structure, the presence of similar structures such as microaneurysms and hemorrhages, micro-vessel with only one to several pixels wide, and requirements for finer results. Methods In this paper, we present a multi-scale deeply supervised network with short connections (BTS-DSN) for vessel segmentation. We used short connections to transfer semantic information between side-output layers. Bottom-top short connections pass low level semantic information to high level for refining results in high-level side-outputs, and top-bottom short connection passes much structural information to low level for reducing noises in low-level side-outputs. In addition, we employ cross-training to show that our model is suitable for real world fundus images. Results The proposed BTS-DSN has been verified on DRIVE, STARE and CHASE_DB1 datasets, and showed competitive performance over other state-of-the-art methods. Specially, with patch level input, the network achieved 0.7891/0.8212 sensitivity, 0.9804/0.9843 specificity, 0.9806/0.9859 AUC, and 0.8249/0.8421 F1-score on DRIVE and STARE, respectively. Moreover, our model behaves better than other methods in cross-training experiments. Conclusions BTS-DSN achieves competitive performance in vessel segmentation task on three public datasets. It is suitable for vessel segmentation. The source code of our method is available at: https://github.com/guomugong/BTS-DSN . <s> BIB007 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> This paper presents a novel two-stage vessel segmentation framework applied to retinal fundus images. In the first stage a convolutional neural network (CNN) is used to correlate an image patch with a corresponding groundtruth reduced using Totally Random Trees Embedding. In the second stage training patches are forward propagated through CNN to create a visual codebook. The codebook is used to build a generative nearest neighbour search space that can be queried by feature vectors created through forward propagating previously-unseen patches through CNN. The proposed framework is able to generate segmentation patches that were not seen during training. Evaluated using publicly available datasets (DRIVE, STARE) demonstrated better performance than state-of-the-art methods in terms of multiple evaluation metrics. The accuracy, robustness, speed and simplicity of the proposed framework demonstrates its suitability for automated vessel segmentation. <s> BIB008 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> Accurate segmentation of vessels is an arduous task in the analysis of medical images, particularly the extraction of vessels from colored retinal fundus images. Many image processing tactics have been implemented for accurate detection of vessels, but many vessels have been dropped. In this paper, we propose a deep learning method based on the convolutional neural network (CNN) with dice loss function for retinal vessel segmentation. To our knowledge, we are the first to form the CNN on the basis of the dice loss function for the extraction of vessels from a colored retinal image. The pre-processing steps are used to eliminate uneven illumination to make the training process more efficient. We implement the CNN model based on a variational auto-encoder (VAE), which is a modified version of U-Net. Our main contribution to the implementation of CNN is to replace all pooling layers with progressive convolution and deeper layers. It takes the retinal image as input and generates the image of segmented output vessels with the same resolution as the input image. The proposed segmentation method showed better performance than the existing methods on the most used databases, namely: DRIVE and STARE. In addition, it gives a sensitivity of 0.739 on the DRIVE database with an accuracy of 0.948 and a sensitivity of 0.748 on the STARE database with an accuracy of 0.947. <s> BIB009 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> FIGURE 19. <s> Automatic retinal vessel segmentation is a fundamental step in the diagnosis of eye-related diseases, in which both thick vessels and thin vessels are important features for symptom detection. All existing deep learning models attempt to segment both types of vessels simultaneously by using a unified pixel-wise loss that treats all vessel pixels with equal importance. Due to the highly imbalanced ratio between thick vessels and thin vessels (namely the majority of vessel pixels belong to thick vessels), the pixel-wise loss would be dominantly guided by thick vessels and relatively little influence comes from thin vessels, often leading to low segmentation accuracy for thin vessels. To address the imbalance problem, in this paper, we explore to segment thick vessels and thin vessels separately by proposing a three-stage deep learning model. The vessel segmentation task is divided into three stages, namely thick vessel segmentation, thin vessel segmentation, and vessel fusion. As better discriminative features could be learned for separate segmentation of thick vessels and thin vessels, this process minimizes the negative influence caused by their highly imbalanced ratio. The final vessel fusion stage refines the results by further identifying nonvessel pixels and improving the overall vessel thickness consistency. The experiments on public datasets DRIVE, STARE, and CHASE_DB1 clearly demonstrate that the proposed three-stage deep learning model outperforms the current state-of-the-art vessel segmentation methods. <s> BIB010
Result of Tan et al BIB003 . From left to right: input images, ground truth, segmentation images and the normalized images. than the above method  . They also measured and reported sensitivity, specificity and accuracy. Their method gave a good sensitivity of 0.75 but the accuracy drops. The reason of good sensitivity was the use of local contrast normalization technique but their method still suffers from losing the tiny vessels as shown in Figure 19 . They did not validate their method on pathologies images. Soomro et al. BIB004 proposed method based CNN along with pre-and-post processing. The performance on both DRIVE and STARE is sensible because they mostly focused on the detection of tiny vessels. They achieved the sensitivity of 75% on both DRIVE and STARE database along with average around 95% accuracy on both databases. Their method BIB005 . From top to bottom: the input images, the ground truth, CNN with original parameters, CNN with FCLs quantization and CNN with FCLs quantization and FCLs pruning. can be improved further for detecting tiny vessels as some of the vessels are dropped by using improved CNN model. They reported all measurement parameters that helps to analyze retinal images properly, especially as they validated their method on challenging images as shown in Figure 20 . Li et al. BIB006 used a database containing retinal fundus images of 5620 patients to validate their method for segmentation of the retinal blood vessels. The proposed method achieved the optimum accuracy of 0.949, but tiny vessels are missing, as shown in Figure 21 , just as they have not compared their method to other existing methods. Guo et al. BIB007 validated their proposed method on the DRIVE database and obtained the best performances compared to the other existing methods. Their method was able to segment the tiny vessels as shown in the Figure 22 and overall gave the best performance with a specificity of 0.9802, a sensitivity of 0.789, an accuracy of 0.956 and an AUC of 0.9802. BIB010 . From top to bottom: the input images, manual annotations, probability maps of thick vessels, probability maps of thin vessels, final probability cards and corresponding hard segmentation images of Yan's method. Chudzik et al. BIB008 validated their method on the DRIVE and STARE databases, which performed well with a sensitivity of 0.788 and an AUC of 0.964 on the DRIVE database, and a sensitivity of 0.826 and an AUC of 0.983 on the STARE database. However, the accuracy was not reported and no visualized segmented images were shown. It is very difficult to analyze the tiny vessels and predict the robustness of the method without observing the segmented images and analyze the accuracy. Hajabdollahi et al. BIB005 validated their method on one of the most challenging databases called STARE database and obtained good performances with an accuracy of 0.961 and AUC of 0.97. The proposed method allows to segment small vessels as shown in the Figure 23 . This method can be considered as one of the source codes for the detection of portable retinal vessels. Yan et al. BIB010 have validated the results of their method on three databases DRIVE, STARE and CHASE-DB1, and their method can detect tiny small vessels and also works in the presence of pathologies, as shown in Figure 24 . The performance of their method on DRIVE and STARE is shown in Table 2 against other existing methods. The performance of their method on the CHASE-DB1 database was a sensitivity of 0.7641, a specificity of 0.9806, a precision of 0.9607 and an AUC of 0.9776. Soomro et al. BIB009 also validated its method on the DRIVE and STARE databases, and their method gave a performance comparable to that of other existing methods. But it requires a lot of improvements because their method does not detect the thin and appropriate vessels and gives a shadow to the optic disc, as shown in Figure 25 , and because of the optic disc shadow, the overall performance of their method is affected. From the analysis of the above methods, we compare the capability of each method in terms of detection of accurate blood vessels as well as the performance of detection of tiny vessels. As per our analysis, each method makes a good contribution for detecting retinal vessels only over certain aspects. We would like to recommend that a good methodology needs three steps to detect proper vessels. First, a preprocessing step is proposed to handle the issue of varying low contrast, such as image contrast and normalization technique. Second, a careful design of CNN should be considered because of the nature of training images. The third step is optional. If there are noisy pixels then post-processing is required otherwise this step can be omitted. We recommend these as on basis of analysis of methodology and FIGURE 25. Toufique et al BIB009 . From left to right: the input images, the output images, the corresponding ground truth images. Top: DRIVE image. Bottom: STARE image. performance of methods. The methods such Fu et al. BIB001 , Liskowskiet al. BIB002 , Soomro et al. BIB004 , Guoet al. BIB007 and Yanet al. BIB010 achieve good performance based on CNN models along with pre-and-post processing steps. In next section, we highlight the main issues and provide more comparison of existing methods for future work contribution in this area.