Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> In previous work we presented a system, MULTIVIEW, used to generate 3D building hypotheses starting from sparse features, hypothesized building corners, extracted from multiple views of the scene. This technique results on knowledge about the imaging geometry and acquisition parameters to provide rigorous geometric constraints for the matching process. The effectiveness of this approach was demonstrated using complex aerial imagery taken with highly oblique views containing buildings with flat or peaked roofs. The MULTIVIEW system uses multiple views based on the successive incorporation of new image data into an existing partial solution. This approach allows the generation of data not seen in the initial images and increases the 3D positioning accuracy of derived object models by simultaneous solution of the collinearity equations. In this paper we provide a detailed performance analysis on the 3D buildings constructed by MULTIVIEW. We evaluate these results with respect to several issues including: (1) Metric accuracy of building recovery. (2) The ability to improve detection and delineation as the number of views are increased. (3) The effect of image processing order on building detection and delineation. The implications of incremental construction of detailed 3D structures are examined with respect to manually derived ground truth data. <s> BIB001 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> An approach is presented for imposing generic hard constraints on deformable models at a low computational cost, while preserving the good convergence properties of snake-like models. We believe this capability to be essential not only for the accurate modeling of individual objects that obey known geometric and semantic constraints but also for the consistent modeling of sets of objects. <s> BIB002 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> We present a surface texture and microstructure extraction system to provide added realism in visualization and virtual reality applications. The system uses multiple images, 3-D models and camera information, in addition to knowledge about man-made structures, to cope with problems such as perspective distortion, data deficiency, and corruption caused by shadows and occlusions. Combined with the Ascender site modeling system and scene rendering algorithms, the system is typically useful for urban site model refinement and visualization. <s> BIB003 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> A scene analysis system for the 3-D modeling of objects is presented. It combines surface reconstruction techniques with object recognition for the generation of 3-D models for computer graphic applications. The system permits the insertion of highlevel constraints, like a specific angle between two house walls, in an explicit knowledge base implemented as a semantic net. The applicability of those constraints is proved by asserting and testing hypotheses in an interpretation phase. In the case of rejection a more general constraint or model is selected. The capabilities of the system were shown for the modeling of buildings using depth from stereo and contour information. The system reconstructs the surface of the scene objects using the constraints selected in the prior interpretation. <s> BIB004 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> A semi automatic approach to three dimensional reconstruction of buildings from aerial imagery is presented. It is based on least squares matching. Two or more images with known interior and exterior orientation are used. Matching is done image-to-image, but the perspective geometry and orientation data are rigorously accounted for. The approach relies on interactive monoscopic measurements in one image. Reconstruction is performed through matching one or more images with the one, where the measurements were done. The matching determines directly object planes for the regions. <s> BIB005 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> This paper describes a system for analysis of aerial images of urban areas using multiple images from different viewpoints and its evaluation. The proposed approach combines bottom-up and top-down processing. In this paper the emphasis is on the discussion of the experimental evaluation. To evaluate statistically the performance of the system, a set of 100 realisations of 5 images from different viewpoints was used, which was generated by combining real and ray-traced images. The experiments show a significant improvement of reliability and accuracy if multi-view imagery is used instead of single-view. <s> BIB006 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> This paper proposes an approach for automatic road extraction in aerial imagery which exploits the scale-space behavior of roads in combination with geometric constrained snake-based edge extraction. The approach not only has few parameters to be adjusted, but for the first time allows for a bridging of shadows and partially occluded areas using the heavily disturbed evidence in the image. The road network is constructed after extracting crossings of various shape and topology. Reasonable results are obtained which are evaluated based on ground truth. <s> BIB007 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> Many tasks in modern urban planning require 3-dimensional (3D) spatial information, preferably in the form of 3D city models. Constructing such models requires automatic methods for reliable 3D building reconstruction. House roofs encountered in residential areas in European cities exhibit a wide variety in their shapes. This limits the use of predefined roof models for their reconstruction. The strategy put forward in this paper is, first, to construct a polyhedral model of the roof structure, which captures the topology of the roof, but which might not be very accurate in a metric sense; and then, in a second step, to improve the metric accuracy by fitting this model to the data. This decoupling of topology extraction from metric reconstruction allows a more efficient roof modelling involving less criteria. And, restricting the processing, at all stages, to one or just a few roof structures, by using a colour-based segmentation of the images, allows to use constraints that are not very tight. The approach has been tested on a state-of-the-art dataset of aerial images of residential areas in Brussels. <s> BIB008 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> This paper addresses two major issues: 3-D building reconstruction and the role of color attributes and similarity grouping. We present ARUBA, a general framework for automated 3-D building reconstruction from multiple color aerial images. After highlighting the strategy and concisely describing the framework and its 2-D and 3-D processing modules, we will evaluate the reconstructed roofs with respect to accurate reference data. The second part of the paper shows that geometry, although important, should not be the only source of information exploited in the reconstruction process. The main objectives are to demonstrate that (1) color is a very important cue in reconstructing a general class of objects, (2) it is crucial to retain all information during the entire processing chain, (3) a general class of objects parts can be efficiently extracted by grouping edges and lines by means of similarity, and (4) a mutual interaction between 2-D and 3-D processing is important. <s> BIB009 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> The Ascender system acquires, extends, and refines 3D geometric site models from calibrated aerial imagery. To acquire a new site model, an automated building detector is run on one image to hypothesize potential building rooftops. Supporting evidence is located in other images via epipolar line segment matching in constrained search regions. The precise 3D shape and location of each building is then determined by multiimage triangulation under geometric constraints of 3D orthogonality, parallelness, colinearity, and coplanarity of lines and surfaces. Projective mapping of image intensity information onto these polyhedral building models results in a realistic site model that can be rendered using virtual “fly-through” graphics. As new images of the site become available, model extension and refinement procedures are performed to add previously unseen buildings and to improve the geometric accuracy of the existing 3D building models. In this way, the system gradually accumulates evidence over time to make the site model more complete and more accurate. An extensive performance evaluation of component algorithms and the full system has been carried out. Two-dimensional building detection accuracy, as well as accuracy of the three-dimensional building reconstruction, are presented for a representative data set. <s> BIB010 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Hypotheses Generation and Search/Resegmentation Based on Spatial Context <s> A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest. <s> BIB011
When Focusing on Contexts    BIB003   improves the probability of hypotheses. Here, it is assumed that part structures cannot be extracted directly in many cases. However, if there is a hypothesis about the object to be extracted, its spatial constraint makes the extraction of the part structures possible. For buildings, single objects such as doors or window cannot be interpreted by themselves (e.g., black blobs), but their arrangement make their semantics and, at the same time, the semantics of the object itself clear. BIB002 BIB007 BIB008  ] enables a geometric improvement of objects with already clear semantics, but weaklydefined outlines. Typical examples are snakes BIB011 or "model-based optimization" BIB002 . Recent results on the extraction of roads in shadowed regions BIB007 show that snakes are also useful to extract objects when only a stabilized geometry makes the extraction of useful image features possible.   BIB009 BIB008 comprises for the data not only color but also multi-spectral images and images from different sensors. Although, the color in images is not stable due to the indirect lighting of shadowed objects, color images are for instance useful for limiting the search space of building extraction by using the fact that many roofs are red. Additionally, there can be, more or less, unexpected colors of the roofs,  BIB009 BIB008 BIB010 BIB001 BIB004 BIB005 BIB006 valuable information about the 3D-geometry of parts of buildings can be gained especially when using Automatic Object Extraction from Aerial Imagery 17 many images from different directions. To get a good approximation, and therefore to improve the probability of the matching, a DSM should be used when available. Methods to produce dense and reliable DSM for urban areas by image matching are, e.g.,   .