LUNGx Challenge for computerized lung nodule classification: reflections and lessons learned.
Challenges, in the context of medical imaging, are valuable in that they allow for a direct comparison of different algorithms designed for a specific radiologic task, with all algorithms abiding by the same set of rules, operating on a common set of images, and being evaluated with a uniform performance assessment paradigm. The variability of system performance based on database composition and subtlety, definition of "truth," and scoring metric is well known;1 3 challenges serve to level the differences across these various dimensions. The medical imaging community has hosted a number of successful thoracic imaging challenges that have spanned a wide range of tasks,4,5 including lung nodule detection,6 lung nodule change, vessel segmentation,7 and vessel tree extraction.8 Each challenge presents its own unique set of circumstances and considerations; however, important common themes exist. Future challenge organizers (and participants) could benefit from an open discussion of successes achieved, pitfalls encountered, and lessons learned from each completed challenge.
Journal of medical imaging