We present an automatic method to describe clinically useful information about scanning, and to guide image interpretation in ultrasound (US) videos of the fet al heart. Our method is able to jointly predict the visibility, viewing plane, location and orientation of the fet al heart at the frame level. The contributions of the paper are three-fold: (i) a convolutional neural network architecture is developed for a multi-task prediction, which is computed by sliding a 3x3 window spatially through convolutional maps. (ii) an anchor mechanism and Intersection over Union (IoU) loss are applied for improving localization accuracy. (iii) a recurrent architecture is designed to recursively compute regional convolutional features temporally over sequential frames, allowing each prediction to be conditioned on the whole video. This results in a spatial-temporal model that precisely describes detailed heart parameters in challenging US videos. We report results on a real-world clinical dataset, where our method achieves performance on par with expert annotations.
The most closely related work to this paper, is that of Bridge @cite_8 , where a number of key parameters related to the fet al heart are estimated. Hand-crafted features were used with classification forests to distinguish different view planes of the fet al heart. Then a multi-variable prediction was formulated that used a CRF-filter. We adress a similar task, but propose a significantly different approach that builds on recent advances in deep learning. Firstly, this allows it to leverage deep, learned feature representations that are shared by all tasks, rather than relying on hand-crafted features. Secondly, by including a recurrent part, it is possible to train our model end-to-end, whereas in @cite_8 the classification and regression parts are trained separately from the temporal filter.
@cite_8
2555486686
Abstract Interpretation of ultrasound videos of the fet al heart is crucial for the antenatal diagnosis of congenital heart disease (CHD). We believe that automated image analysis techniques could make an important contribution towards improving CHD detection rates. However, to our knowledge, no previous work has been done in this area. With this goal in mind, this paper presents a framework for tracking the key variables that describe the content of each frame of freehand 2D ultrasound scanning videos of the healthy fet al heart. This represents an important first step towards developing tools that can assist with CHD detection in abnormal cases. We argue that it is natural to approach this as a sequential Bayesian filtering problem, due to the strong prior model we have of the underlying anatomy, and the ambiguity of the appearance of structures in ultrasound images. We train classification and regression forests to predict the visibility, location and orientation of the fet al heart in the image, and the viewing plane label from each frame. We also develop a novel adaptation of regression forests for circular variables to deal with the prediction of cardiac phase. Using a particle-filtering-based method to combine predictions from multiple video frames, we demonstrate how to filter this information to give a temporally consistent output at real-time speeds. We present results on a challenging dataset gathered in a real-world clinical setting and compare to expert annotations, achieving similar levels of accuracy to the levels of inter- and intra-observer variation.