Real time intraoperative diagnosis by deep neural network driven multiphoton virtual histology.
Recent advances in label free virtual histology promise a new era for real time molecular diagnosis in the operating room and during biopsy procedures. To take full advantage of the rich, multidimensional information provided by these technologies, reproducible and reliable computational tools that could facilitate the diagnosis are in great demand. In this study, we developed a deep learning based framework to recognize cancer versus normal human breast tissue from real time label free virtual histology images, with a tile level AUC (area under receiver operating curve) of 95% and slide level AUC of 100% on unseen samples. Furthermore, models trained on a high quality laboratory generated dataset can generalize to independent datasets acquired from a portable intraoperative version of the imaging technology with a physics based adapted design. Classification activation maps and final feature visualization revealed discriminative patterns, such as tumor cells and tumor associated vesicles, that are highly associated with cancer status. These results demonstrate that through the combination of real time virtual histopathology and a deep learning framework, accurate real time diagnosis could be achieved in point of procedure clinical applications.
NPJ precision oncology