Multi modal automatic montaging of adaptive optics retinal images
We present a fully automated adaptive optics (AO) retinal image montaging algorithm using classic scale invariant feature transform with random sample consensus for outlier removal. Our approach is capable of using information from multiple AO modalities (confocal, split detection, and dark field) and can accurately detect discontinuities in the montage. The algorithm output is compared to manual montaging by evaluating the similarity of the overlapping regions after montaging, and calculating the detection rate of discontinuities in the montage. Our results show that the proposed algorithm has high alignment accuracy and a discontinuity detection rate that is comparable (and often superior) to manual montaging. In addition, we analyze and show the benefits of using multiple modalities in the montaging process. We provide the algorithm presented in this paper as open source and freely available to download. (c) 2016 Optical Society of America OCIS codes: (100.0100) Image processing; (170.4470) Ophthalmology; (110.1080) Active or adaptive optics. References and links 1. J. Liang, D. R. Williams, and D. T. Miller, "Supernormal vision and high resolution retinal imaging through adaptive optics," J. Opt. Soc. Am. 14, 2884 2892 (1997) 2. A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams, "Packing arrangement of the three cone classes in primate retina," Vision Res. 41, 1291 1306 (2001) 3. A. Dubra, Y. Sulai, J. L. Norris, R. F. Cooper, A. M. Dubis, D. R. Williams, and J. Carroll, "Noninvasive imaging of the human rod photoreceptor mosaic using a confocal adaptive optics scanning ophthalmoscope," Biomed. Opt. Express 2, 1864 1876 (2011) 4. J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, "In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic," Invest. Ophthalmol. Vis. Sci. 50, 1350 1359 (2009) 5. A. Roorda, F. Romero Borja, W. Donnelly III, H. Queener, T. Hebert, and M. Campbell, "Adaptive optics scanning laser ophthalmoscopy," Opt. Express 10, 405 412 (2002) 6. T. Y. Chui, D. A. VanNasdale, and S. A. Burns, "The use of forward scatter to improve retinal vascular imaging with an adaptive optics scanning laser ophthalmoscope," Biomed. Opt. Express 3, 2537 2549 (2012) 7. J. I. Morgan, "The fundus photo has met its match: optical coherence tomography and adaptive optics ophthalmoscopy are here to stay," Ophthalmic. Physiol. Opt. 36, 218 239 (2016) 8. H. Li, J. Lu, G. Shi, and Y. Zhang, "Automatic montage of retinal images in adaptive optics confocal scanning laser ophthalmoscope," Opt. Eng. 51, 057008 1 (2012) 9. Y. Ke and R. Sukthankar, "PCA SIFT: A more distinctive representation for local image descriptors," in "Proc. of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition," vol. 2 (IEEE, 2004) vol. 2, pp. II 506. 10. D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, "In vivo imaging of human cone photoreceptor inner segments," Invest. Ophthalmol. Vis. Sci. 55, 4244 4251 (2014) 11. D. Scoles, Y. N. Sulai, and A. Dubra, "In vivo dark field imaging of the retinal pigment epithelium cell mosaic," Biomed. Opt. Express 4, 1710 1723 (2013) 12. A. Standard, "American national standard for the safe use of lasers. American National Standards Institute," Inc. New York (1993) 13. A. Dubra and Y. Sulai, "Reflective afocal broadband adaptive optics scanning ophthalmoscope," Biomed. Opt. Express 2, 1757 1768 (2011) 14. A. Dubra and Z. Harvey, "Registration of 2D images from fast scanning ophthalmic instruments," in "Proc. of 2010 International Workshop on Biomedical Image Registration," (Springer, 2010) pp. 60 71. Vol. 7, No. 12 1 Dec 2016 BIOMEDICAL OPTICS EXPRESS 4899 #274955 http:/dx.doi.org/10.1364/BOE.7.004899 Journal (c) 2016 Received 1 Sep 2016; revised 21 Oct 2016; accepted 24 Oct 2016; published 3 Nov 2016 15. D. G. Lowe, "Object recognition from local scale invariant features," in "Proc. of 1999 IEEE international Conference on Computer Vision," vol. 2 (Ieee, 1999) vol. 2, pp. 1150 1157. 16. M. A. Fischler and R. C. Bolles, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM 24, 381 395 (1981) 17. B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, "Symmetric diffeomorphic image registration with cross correlation: evaluating automated labeling of elderly and neurodegenerative brain," Med. Image Anal. 12, 26 41 (2008) 18. A. Sotiras, C. Davatzikos, and N. Paragios, "Deformable medical image registration: A survey," IEEE Trans. Med. Imag. 32, 1153 1190 (2013) 19. A. Strehl and J. Ghosh, "Cluster ensembles a knowledge reuse framework for combining multiple partitions," J. Mach. Learn. Res. 3, 583 617 (2002) 20. C. E. Shannon, "A mathematical theory of communication," ACM SIGMOBILE Mobile Computing and Communications Review 5, 3 55 (2001) 21. J. P. Pluim, J. A. Maintz, and M. A. Viergever, "Mutual information based registration of medical images: a survey," IEEE Trans. Med. Imag. 22, 986 1004 (2003) 22. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "Speeded up robust features (SURF)" Comput. Vis. Image Und. 110, 346 359 (2008) 23. J. M. Morel and G. Yu, "Asift: A new framework for fully affine invariant image comparison," SIAM J. Imaging Sci. 2, 438 469 (2009) 24. J. C. Niebles, H. Wang, and L. Fei Fei, "Unsupervised learning of human action categories using spatial temporal words," Int. J. Comput. Vision 79, 299 318 (2008)