SyntheX: Scaling Up Learning-based X-ray Image Analysis Through In Silico Experiments
Artificial intelligence (AI) now enables automated interpretation of medical
images for clinical use. However, AI's potential use for interventional images
(versus those involved in triage or diagnosis), such as for guidance during
surgery, remains largely untapped. This is because surgical AI systems are
currently trained using post hoc analysis of data collected during live
surgeries, which has fundamental and practical limitations, including ethical
considerations, expense, scalability, data integrity, and a lack of ground
truth. Here, we demonstrate that creating realistic simulated images from human
models is a viable alternative and complement to large-scale in situ data
collection. We show that training AI image analysis models on realistically
synthesized data, combined with contemporary domain generalization or
adaptation techniques, results in models that on real data perform comparably
to models trained on a precisely matched real data training set. Because
synthetic generation of training data from human-based models scales easily, we
find that our model transfer paradigm for X-ray image analysis, which we refer
to as SyntheX, can even outperform real data-trained models due to the
effectiveness of training on a larger dataset. We demonstrate the potential of
SyntheX on three clinical tasks: Hip image analysis, surgical robotic tool
detection, and COVID-19 lung lesion segmentation. SyntheX provides an
opportunity to drastically accelerate the conception, design, and evaluation of
intelligent systems for X-ray-based medicine. In addition, simulated image
environments provide the opportunity to test novel instrumentation, design
complementary surgical approaches, and envision novel techniques that improve
outcomes, save time, or mitigate human error, freed from the ethical and
practical considerations of live human data collection.