Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> A computational method is reported which allows the fully automated recovery of the three-dimensional shape of the cardiac left ventricle from a reduced set of apical echo views. Two typically ill-posed problems have been faced: 1) the detection of the left ventricle contours in each view, and 2) the integration of the detected contour points (which form a sparse and partially inconsistent data set) into a single surface representation. The authors' solution to these problems is based on a careful integration of standard computer vision algorithms with neural networks. Boundary detection comprises three steps: edge detection, edge grouping, and edge classification. The first and second steps (which are typical early-vision tasks not involving specific domain-knowledge) have been performed through fast, well-established algorithms of computer vision. The higher level task of left ventricle-edge discrimination, which involves the exploitation of specific knowledge about the left ventricle silhouette, has been performed by feedforward neural networks. Following the most recent results in the field of computer vision, the first step in solving the problem of recovering the ventricle surface has been the adoption of a physically inspired model of it. Basically, the authors have modeled the left ventricle surface as a closed, thin, elastic surface and the data as a set of radial springs acting on it. The recovery process is equivalent to the settling of the surface-plus-springs system into a stable configuration of minimum potential energy. The finite element discretization of this model leads directly to an analog neural-network implementation. The efficiency of such an implementation has been remarkably enhanced through a learning algorithm which embeds specific knowledge about the shape of the left ventricle in the network. Experiments using clinical echographic sequences are described. Four apical views (each with a different rotation of the probe) have been acquired during a heartbeat from a set of seven normal subjects. These images have been utilized to set the various processing modules and test their capabilities. > <s> BIB001 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> We study a model of multiscale analysis (or scale-space) applied to movies. This model comes from a thorough formalization that has been done in the theory of scale-space of static image. This formulation has led to associate with each multiscale analysis a partial differential equation (PDE). We intend here to examine the case of movies, and to insist on the motion aspects. More precisely, it has been proved in that there exists a unique affine and morphological and Galilean invariant scale-space for movies, the AMG model. This model is described by a partial differential equation. In this paper,we focus on terms appearing in that equation. We show that this model provides a reliable definition of an optical multiscale acceleration. At the practical level, scale is interpreted as a way of characterizing reliable trajectories. As we prove by experiments,the AMG model is a riddle for decimating spurious trajectories due to any kinds of nonadditive impurities and noise. Simple discrete formulae are given to implement the model. <s> BIB002 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> Introduces a new model for multiscale analysis of space-time echocardiographic sequences. The proposed nonlinear partial differential equation, representing the multiscale analysis, filters the sequence while keeping the space-time coherent structures. It combines the ideas of regularized Perona-Malik anisotropic diffusion and the Galilean invariant movie multiscale analysis of Alvarez et al. (Arch. Rat. Mech. Anal., vol. 123, p. 200-57, 1993). A numerical method for solving the proposed partial differential equation is suggested and its stability is shown. Computational results on synthesized and real sequences are provided. A qualitative and quantitative evaluation of the accuracy of the method is presented. <s> BIB003 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> We present a model and algorithm for segmentation of images with missing boundaries. In many situations, the human visual system fills in missing gaps in edges and boundaries, building and completing information that is not present. This presents a considerable challenge in computer vision, since most algorithms attempt to exploit existing data. Completion models, which postulate how to construct missing data, are popular but are often trained and specific to particular images. In this paper, we take the following perspective: We consider a reference point within an image as given and then develop an algorithm that tries to build missing information on the basis of the given point of view and the available information as boundary data to the algorithm. We test the algorithm on some standard images, including the classical triangle of Kanizsa and low signal/noise ratio medical images. <s> BIB004 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> We present a geometric model and a computational method for segmentation of images with missing boundaries. In many situations, the human visual system fills in missing gaps in edges and boundaries, building and completing information that is not present. Boundary completion presents a considerable challenge in computer vision, since most algorithms attempt to exploit existing data. A large body of work concerns completion models, which postulate how to construct missing data; these models are often trained and specific to particular images. In this paper, we take the following, alternative perspective: we consider a reference point within an image as given, and then develop an algorithm which tries to build missing information on the basis of the given point of view and the available information as boundary data to the algorithm. Starting from this point of view, a surface is constructed. It is then evolved with the mean curvature flow in the metric induced by the image until a piecewise constant solution is reached. We test the computational model on modal completion, amodal completion, texture, photo and medical images. We extend the geometric model and the algorithm to 3D in order to extract shapes from low signal/noise ratio medical volumes. Results in 3D echocardiography and 3D fetal echography are presented. <s> BIB005 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> This paper presents a method of four-dimensional (4-D) (3-D+Time) space-frequency analysis for directional denoising and enhancement of real-time three-dimensional (RT3D) ultrasound and quantitative measures in diagnostic cardiac ultrasound. Expansion of echocardiographic volumes is performed with complex exponential wavelet-like basis functions called brushlets. These functions offer good localization in time and frequency and decompose a signal into distinct patterns of oriented harmonics, which are invariant to intensity and contrast range. Deformable-model segmentation is carried out on denoised data after thresholding of transform coefficients. This process attenuates speckle noise while preserving cardiac structure location. The superiority of 4-D over 3-D analysis for decorrelating additive white noise and multiplicative speckle noise on a 4-D phantom volume expanding in time is demonstrated. Quantitative validation, computed for contours and volumes, is performed on in vitro balloon phantoms. Clinical applications of this spatiotemporal analysis tool are reported for six patient cases providing measures of left ventricular volumes and ejection fraction. <s> BIB006 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> The major obstacle of three-dimensional (3-D) echocardiography is that the ultrasound image quality is too low to reliably detect features locally. Almost all available surface-finding algorithms depend on decent quality boundaries to get satisfactory surface models. We formulate the surface model optimization problem in a Bayesian framework, such that the inference made about a surface model is based on the integration of both the low-level image evidence and the high-level prior shape knowledge through a pixel class prediction mechanism. We model the probability of pixel classes instead of making explicit decisions about them. Therefore, we avoid the unreliable edge detection or image segmentation problem and the pixel correspondence problem. An optimal surface model best explains the observed images such that the posterior probability of the surface model for the observed images is maximized. The pixel feature vector as the image evidence includes several parameters such as the smoothed grayscale value and the minimal second directional derivative. Statistically, we describe the feature vector by the pixel appearance probability model obtained by a nonparametric optimal quantization technique. Qualitatively, we display the imaging plane intersections of the optimized surface models together with those of the ground-truth surfaces reconstructed from manual delineations. Quantitatively, we measure the projection distance error between the optimized and the ground-truth surfaces. In our experiment, we use 20 studies to obtain the probability models offline. The prior shape knowledge is represented by a catalog of 86 left ventricle surface models. In another set of 25 test studies, the average epicardial and endocardial surface projection distance errors are 3.2 +/- 0.85 mm and 2.6 +/- 0.78 mm, respectively. <s> BIB007 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> Echocardiography (cardiac ultrasound) is today the predominant technique for quantitative assessment of cardiac function and valvular heart lesions. Segmentation of cardiac structures is required to determine many important diagnostic parameters. As the heart is a moving organ, reliable information can be obtained only from three-dimensional (3-D) data over time (3-D + time = 4-D). Due to their size, the resulting four-dimensional (4-D) data sets are not reasonably accessible to simple manual segmentation methods. Automatic segmentation often yields unsatisfactory results in a clinical environment, especially for ultrasonic images. We describe a semiautomated segmentation algorithm (ROPES) that is able to greatly reduce the time necessary for user interaction and its application to extract various parameters from 4-D echocardiographic data. After searching for candidate contour points, which have to fulfill a multiscale edge criterion, the candidates are connected by minimizing a cost function to line segments that then are connected to form a closed contour. The contour is automatically checked for plausibility. If necessary, two correction methods that can also be used interactively are applied (fitting of other line segments into the contour and searching for additional candidates with a relaxed criterion). The method is validated using in vivo transesophageal echocardiographic data sets. <s> BIB008 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> A morphological multiscale method in 3D image and 3D image sequence processing is discussed which identifies edges on level sets and the motion of features in time. Based on these indicator evaluation the image data is processed applying nonlinear diffusion and the theory of geometric evolution problems. The aim is to smooth level sets of a 3D image while preserving geometric features such as edges and corners on the level sets and to simultaneously respect the motion and acceleration of object in time. An anisotropic curvature evolution is considered in space. Whereas, in case of an image sequence a weak coupling of these separate curvature evolutions problems is incorporated in the time direction of the image sequence. The time of the actual evolution problem serves as the multiscale parameter. The spatial diffusion tensor depends on a regularized shape operator of the evolving level sets and the evolution speed is weighted according to an approximation of the apparent acceleration of objects. As one suitable regularization tool local L 2—projection onto polynomials is considered. A spatial finite element discretization on hexahedral meshes, a semi-implicit, regularized backward Euler discretization in time, and an explicit coupling of subsequent images in case of image sequences are the building blocks of the algorithm. Different applications underline the efficiency of the presented image processing tool. <s> BIB009 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> A new morphological multiscale method in three-dimensional (3D) image processing is presented which combines the image processing methodology based on nonlinear diffusion equations and the theory of geometric evolution problems. Its aim is to smooth the level sets of a 3D image while simultaneously preserving geometric features such as edges and corners on the level sets. This is obtained by an anisotropic curvature evolution, where time serves as the multiscale parameter. Thereby the diffusion tensor depends on a regularized shape operator of the evolving level sets. As one suitable regularization, local L2 projection onto quadratic polynomials is considered. The method is compared to a related parametric surface approach, and a geometric interpretation of the evolution and its invariance properties is given. A spatial finite element discretization on hexahedral meshes and a semi-implicit, regularized backward Euler discretization in time are the building blocks of the easy-to-code algorithm. Different a... <s> BIB010 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> The application of level set techniques to echocardiographic data is presented. This method allows semiautomatic segmentation of heart chambers, which regularizes the shapes and improves edge fidelity, especially in the presence of gaps, as is common in ultrasound data. The task of the study was to reconstruct left ventricular shape and to evaluate left ventricular volume. Data were acquired with a real-time three-dimensional (3-D) echocardiographic system. The method was applied directly in the three-dimensional domain and was based on a geometric-driven scheme. The numerical scheme for solving the proposed partial differential equation is borrowed from numerical methods for conservation law. Results refer to in vitro and human in vivo acquired 3-D+time echocardiographic data. Quantitative validation was performed on in vitro balloon phantoms. Clinical application of this segmentation technique is reported for 20 patient cases providing measures of left ventricular volumes and ejection fraction. <s> BIB011 </s> Ultrasound image segmentation: a survey <s> 3) 3-D and 4-D Endocardial Border Detection: <s> We present a morphological multi-scale method for image sequence processing, which results in a truly coupled spatio-temporal anisotropic diffusion. The aim of the method is not to smooth the level-sets of single frames but to denoise the whole sequence while retaining geometric features such as spatial edges and highly accelerated motions. This is obtained by an anisotropic spatio-temporal level-set evolution, where the additional artificial time variable serves as the multi-scale parameter. The diffusion tensor of the evolution depends on the morphology of the sequence, given by spatial curvatures of the level-sets and the curvature of trajectories (=acceleration) in sequence-time. We discuss different regularization techniques and describe an operator splitting technique for solving the problem. Finally we compare the new method with existing multi-scale image sequence processing methodologies. <s> BIB012
In relatively early work on 3-D endocardial border detection, Coppini et al. BIB001 considered segmentation and reconstruction of the left ventricle from freehand ultrasound data where data was acquired at four fixed angles with respect to the four-chamber view. Boundaries were found in each 2-D slice as the zero-crossings of a Laplacian of a Gaussian edge operator. Edge points were then linked to form contours. A neural network was then used to classify edge segments as part of a boundary or not (training was done on data from four subjects, with testing on three others). An elastic surface model was then fit to the edges. Preliminary assessment of the different processing stages was given. Song et al. treated the segmentation problem as a surface fitting problem posed in a Bayesian framework where the goal was to find a 3-D surface which has the greatest posterior probability given the images BIB007 . They argued that by omitting an explicit feature detection step their method is less sensitive to low-quality images. They tested their method using 45 data sets from normal and diseased hearts; 20 of the datasets were used to train the model (determine pixel appearance and pixel class prediction models). The average epicardial and endocardial surface projection distance errors for end-diastolic frames were found to be 3.2 and 2.6 mm, respectively. This is interesting but preliminary work that to our knowledge has not be pursued further. Wolf et al. BIB008 described a semi-automatic method of segmentation called the restricted optimal path exploring segmentation (ROPES) approach which was applied to 2-D slices of 3-D transesophaegal echocardiography (TEE) data. It is based on the idea of first finding candidate edge points which satisfy a multiscale criterion and minimizing a cost function which forms line segments and then closed contours. Comparison is made with manual delineations where a mean distance in-plane error of 3.44 mm was reported (cf. 2.92 mm interobserver distance). Angelini et al. BIB006 considered segmentation of real-time 3-D echocardiography data from a Volumetrics system. First, the data was preprocessed with a wavelet analysis (using brushlets) to reduce speckle. Then a 2-D balloon deformable model, using an intensity gradient based force, was fitted to each slice and the volume derived from this. The method was tested on six clinical datasets with comparison made with MR manual segmentation on end-diastole and end-systole frames. They referenced a paper by Takuma et al.  which claims an interobserver variability of 8.3% and intra-observer variability of 3.7% for volume estimation with disk summation on real-time 3-D (RT3-D) echocardiography data. Their method shows an 8.9% error compared to manual delineation on MR data whereas manual delineation on RT3-D versus manual delineation on MR showed a much higher average value. This paper, as with our own experience, shows that one has to take care in using manual delineation on ultrasound data as a reference. Montagnat et al.  developed a model-based segmentation method for rotational 3-D echocardiography. This work is most interesting for its use of a cylindrical geometry for analysis, and using a combined edge and region-based approach. Four-dimensional (4-D) anisotropic diffusion filtering was first applied to the data to reduce speckle (no distinction was made between temporal and spatial dimensions). This method assumes boundaries are located at high intensity gradients and the parameter in the diffusion process is a function of the intensity gradient magnitude. Thus, this work does not explicitly account for imaging physics. Then a deformable model (discrete simplex mesh) is fit to the filtered data where the external forces are provided by a 3-D gradient operator defined in cylindrical coordinates (the 3-D equivalent of the work of Herlin and Ayache  ). Specifically the intensity gradient magnitude and orientation are used, together with a region-based force based on a local analysis of intensity values. Here, it is perhaps surprising that the anisotropic diffusion was also not done in the natural frame of reference of acquisition. Qualitative results are presented. Although strictly speaking filtering methods, the next two papers are theoretically elegant approaches which have shown promising early results as ultrasound segmentation methods. The solutions are posed in a variational framework and have properties that make them well-suited to 3-D echocardiography segmentation. Sarti et al. BIB003 developed a model for multiple scale analysis of space-time echocardiographic sequences based on solving a nonlinear partial differential equation combining ideas of the Perona-Malik anisotropic diffusion model with the Galilean invariant movie multiscale analysis of Guichard BIB002 ,  . Coherent spatio-temporal structures are preserved by the filtering operation. Their method was applied to a single example of rotational 3-D echocardiographic data. In an extension BIB009 , the Perona-Malik spatial diffusion was replaced with a curvature driven level-sets nonlinear anisotropic smoothing BIB010 . That model was invariant under gray value transformations and, unlike previous curvature evolution models, it wellpreserved edges and corners on level-sets while still allowing tangential smoothing along edges. In later work, Mikula et al. presented a truly coupled spatio-temporal anisotropic diffusion BIB012 . In that work, time smoothing was curvature driven, and the authors argue, resulted in better smoothing with velocity discontinuity preservation in contrast to the previous acceleration based smoothing. Note that, in contrast to the 4-D diffusion model used in  , the models of BIB003 and BIB012 , treat the temporal dimension explicitly as time rather than treating space and time as equivalent dimensions. Corsi et al. BIB011 presented a level set based segmentation approach which was applied to real-time 3-D echocardiography from a Volumetrics system. The classical inflationary term in the speed function was removed in order the avoid leakage. Hence, close initialization of the level set function was required. Three parameters also had to be chosen. The level set was initialized using manual delineation on a few (short axis) frames. The method was tested in vitro (on balloons of various volumes ) with a correlation coefficient of and an underestimation of the true volume. On clinical data a correlation of was found when a comparison was made with MR volumes treated as the reference. A 15.58 ml bias was observed in this case (underestimation by level set method). Sarti et al. presented a level set segmentation algorithm-the subjective surfaces BIB004 , BIB005 -for images, such as ultrasound images, which can have missing boundaries. Their approach differs fundamentally from previous level-set segmentation algorithms which concern only the zero level set. In this work, all the level sets, hence the surface, evolve under a suitably chosen speed function. The flow sharpens the surface around the edges and connects segmented boundaries across the missing information. This work was further extended by Mikula et al.  and Corsaro et al.  . An efficient and robust semi-implicit complementary volume numerical scheme was proposed in order to solve the Riemannian mean curvature flow equation in 2-D  and 3-D  . Segmentation examples of the classical Kanizsa triangle were shown to demonstrate that the method can accommodate missing boundaries. Examples on 3-D cardiac  and fetal ultrasound BIB004 images were also shown.