We present an automatic method to describe clinically useful information about scanning, and to guide image interpretation in ultrasound (US) videos of the fet al heart. Our method is able to jointly predict the visibility, viewing plane, location and orientation of the fet al heart at the frame level. The contributions of the paper are three-fold: (i) a convolutional neural network architecture is developed for a multi-task prediction, which is computed by sliding a 3x3 window spatially through convolutional maps. (ii) an anchor mechanism and Intersection over Union (IoU) loss are applied for improving localization accuracy. (iii) a recurrent architecture is designed to recursively compute regional convolutional features temporally over sequential frames, allowing each prediction to be conditioned on the whole video. This results in a spatial-temporal model that precisely describes detailed heart parameters in challenging US videos. We report results on a real-world clinical dataset, where our method achieves performance on par with expert annotations.
Automatic methods for US video analysis have been developed. For example, Kwitt @cite_2 applied kernel dynamic texture models for labelling individual frames in an US video. This concept was extended to handle multiple object classes on real-world clinical US videos in @cite_4 . Recently, CNNs have been applied to image-level classification of anatomical structures with transfer learning @cite_7 and recurrent models @cite_9 . These works are all related to frame-level classification of US videos. Our application differs from them by focusing on describing details of the fet al heart, which is a more complicated multi-task application.
@cite_9 @cite_4 @cite_7 @cite_2
2294318823 2569161580 2430776200 2063540402
Accurate acquisition of fet al ultrasound US standard planes is one of the most crucial steps in obstetric diagnosis. The conventional way of standard plane acquisition requires a thorough knowledge of fet al anatomy and intensive manual labors. Hence, automatic approaches are highly demanded in clinical practice. However, automatic detection of standard planes containing key anatomical structures from US videos remains a challenging problem due to the high intra-class variations of standard planes. Unlike previous studies that developed specific methods for different anatomical standard planes respectively, we present a general framework to detect standard planes from US videos automatically. Instead of utilizing hand-crafted visual features, our framework explores spatio-temporal feature learning with a novel knowledge transferred recurrent neural network T-RNN, which incorporates a deep hierarchical visual feature extractor and a temporal sequence learning model. In order to extract visual features effectively, we propose a joint learning framework with knowledge transfer across multi-tasks to address the insufficiency issue of limited training data. Extensive experiments on different US standard planes with hundreds of videos corroborate that our method can achieve promising results, which outperform state-of-the-art methods. Abstract Confirmation of pregnancy viability (presence of fet al cardiac activity) and diagnosis of fet al presentation (head or buttock in the maternal pelvis) are the first essential components of ultrasound assessment in obstetrics. The former is useful in assessing the presence of an on-going pregnancy and the latter is essential for labour management. We propose an automated framework for detection of fet al presentation and heartbeat from a predefined free-hand ultrasound sweep of the maternal abdomen. Our method exploits the presence of key anatomical sonographic image patterns in carefully designed scanning protocols to develop, for the first time, an automated framework allowing novice sonographers to detect fet al breech presentation and heartbeat from an ultrasound sweep. The framework consists of a classification regime for a frame by frame categorization of each 2D slice of the video. The classification scores are then regularized through a conditional random field model, taking into account the temporal relationship between the video frames. Subsequently, if consecutive frames of the fet al heart are detected, a kernelized linear dynamical model is used to identify whether a heartbeat can be detected in the sequence. In a dataset of 323 predefined free-hand videos, covering the mother’s abdomen in a straight sweep, the fet al skull, abdomen, and heart were detected with a mean classification accuracy of 83.4 . Furthermore, for the detection of the heartbeat an overall classification accuracy of 93.1 was achieved. We address the task of object recognition in obstetric ultrasound videos using deep Convolutional Neural Networks (CNNs). A transfer learning based design is presented to study the transferability of features learnt from natural images to ultrasound image object recognition which on the surface is a very different problem. Our results demonstrate that CNNs initialised with large-scale pre-trained networks outperform those directly learnt from small-scale ultrasound data (91.5 versus 87.9 ), in terms of object identification. The problem of localizing specific anatomic structures using ultrasound (US) video is considered. This involves automatically determining when an US probe is acquiring images of a previously defined object of interest, during the course of an US examination. Localization using US is motivated by the increased availability of portable, low-cost US probes, which inspire applications where inexperienced personnel and even first-time users acquire US data that is then sent to experts for further assessment. This process is of particular interest for routine examinations in underserved populations as well as for patient triage after natural disasters and large-scale accidents, where experts may be in short supply. The proposed localization approach is motivated by research in the area of dynamic texture analysis and leverages several recent advances in the field of activity recognition. For evaluation, we introduce an annotated and publicly available database of US video, acquired on three phantoms. Several experiments reveal the challenges of applying video analysis approaches to US images and demonstrate that good localization performance is possible with the proposed solution.