Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Specific Parts of the Model for Building Extraction comprise: <s> A new method for extracting planar polygonal rooftops in monocular aerial imagery is proposed. Structural features are extracted and hierarchically related using perceptual grouping techniques. Top-down feature verification is used so that features, and links between the features, are verified with local information in the image and weighed in a graph. Cycles in the graph correspond to possible building rooftop hypotheses. Virtual features are hypothesized for the perceptual completion of partially occluded rooftops. Extraction of the "best" grouping features into a building rooftop hypothesis is posed as a graph search problem. The maximally weighted, independent set of cycles in the graph is extracted as the final set of roof boundaries. > <s> BIB001 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Specific Parts of the Model for Building Extraction comprise: <s> In previous work we presented a system, MULTIVIEW, used to generate 3D building hypotheses starting from sparse features, hypothesized building corners, extracted from multiple views of the scene. This technique results on knowledge about the imaging geometry and acquisition parameters to provide rigorous geometric constraints for the matching process. The effectiveness of this approach was demonstrated using complex aerial imagery taken with highly oblique views containing buildings with flat or peaked roofs. The MULTIVIEW system uses multiple views based on the successive incorporation of new image data into an existing partial solution. This approach allows the generation of data not seen in the initial images and increases the 3D positioning accuracy of derived object models by simultaneous solution of the collinearity equations. In this paper we provide a detailed performance analysis on the 3D buildings constructed by MULTIVIEW. We evaluate these results with respect to several issues including: (1) Metric accuracy of building recovery. (2) The ability to improve detection and delineation as the number of views are increased. (3) The effect of image processing order on building detection and delineation. The implications of incremental construction of detailed 3D structures are examined with respect to manually derived ground truth data. <s> BIB002 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Specific Parts of the Model for Building Extraction comprise: <s> Abstract The paper discusses the modeling necessary for recovering man made objects—in this case buildings—in complex scenes from digital imagery. The approach addresses all levels of image analysis for deriving semantically meaningful descriptions of the scene from the image, via the geometrical/physical model of the objects and their counterparts in the image. The central link between raster image and scene are network-like organized aspects of parts of the objects. This is achieved by generically modeling the objects using parametrized volume primitives together with the application-specific constraints, which seems to be adequate for many types of buildings. The paper sketches the various interrelationships between the different models and their use for feature extraction, hypothesis generation, and verification. <s> BIB003 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Specific Parts of the Model for Building Extraction comprise: <s> We describe a method for detecting rectilinear buildings and constructing their 3-D shape descriptions from a single aerial image of a general viewpoint. 2-D roof hypotheses are generated from linear features by perceptual grouping. Good hypotheses are selected and then verified by computing wall and shadow evidence for them, which also provide the height information for the buildings. A 3-D reasoning process resolves conflicts among hypotheses in 3-D space. Results from several images can be integrated at a high level. An interactive system allows efficient editing of results by making use of the analysis performed by the automatic system; it also allows for some initial preparation of the data to improve results of the automatic system. Some results and their evaluation are included. <s> BIB004 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Specific Parts of the Model for Building Extraction comprise: <s> A scene analysis system for the 3-D modeling of objects is presented. It combines surface reconstruction techniques with object recognition for the generation of 3-D models for computer graphic applications. The system permits the insertion of highlevel constraints, like a specific angle between two house walls, in an explicit knowledge base implemented as a semantic net. The applicability of those constraints is proved by asserting and testing hypotheses in an interpretation phase. In the case of rejection a more general constraint or model is selected. The capabilities of the system were shown for the modeling of buildings using depth from stereo and contour information. The system reconstructs the surface of the scene objects using the constraints selected in the prior interpretation. <s> BIB005 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Specific Parts of the Model for Building Extraction comprise: <s> This paper addresses two major issues: 3-D building reconstruction and the role of color attributes and similarity grouping. We present ARUBA, a general framework for automated 3-D building reconstruction from multiple color aerial images. After highlighting the strategy and concisely describing the framework and its 2-D and 3-D processing modules, we will evaluate the reconstructed roofs with respect to accurate reference data. The second part of the paper shows that geometry, although important, should not be the only source of information exploited in the reconstruction process. The main objectives are to demonstrate that (1) color is a very important cue in reconstructing a general class of objects, (2) it is crucial to retain all information during the entire processing chain, (3) a general class of objects parts can be efficiently extracted by grouping edges and lines by means of similarity, and (4) a mutual interaction between 2-D and 3-D processing is important. <s> BIB006 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Specific Parts of the Model for Building Extraction comprise: <s> Many tasks in modern urban planning require 3-dimensional (3D) spatial information, preferably in the form of 3D city models. Constructing such models requires automatic methods for reliable 3D building reconstruction. House roofs encountered in residential areas in European cities exhibit a wide variety in their shapes. This limits the use of predefined roof models for their reconstruction. The strategy put forward in this paper is, first, to construct a polyhedral model of the roof structure, which captures the topology of the roof, but which might not be very accurate in a metric sense; and then, in a second step, to improve the metric accuracy by fitting this model to the data. This decoupling of topology extraction from metric reconstruction allows a more efficient roof modelling involving less criteria. And, restricting the processing, at all stages, to one or just a few roof structures, by using a colour-based segmentation of the images, allows to use constraints that are not very tight. The approach has been tested on a state-of-the-art dataset of aerial images of residential areas in Brussels. <s> BIB007 </s> Automatic Object Extraction from Aerial Imagery – A Survey Focusing on Buildings Helmut Mayer Chair for Photogrammetry and Remote Sensing, <s> Specific Parts of the Model for Building Extraction comprise: <s> The Ascender system acquires, extends, and refines 3D geometric site models from calibrated aerial imagery. To acquire a new site model, an automated building detector is run on one image to hypothesize potential building rooftops. Supporting evidence is located in other images via epipolar line segment matching in constrained search regions. The precise 3D shape and location of each building is then determined by multiimage triangulation under geometric constraints of 3D orthogonality, parallelness, colinearity, and coplanarity of lines and surfaces. Projective mapping of image intensity information onto these polyhedral building models results in a realistic site model that can be rendered using virtual “fly-through” graphics. As new images of the site become available, model extension and refinement procedures are performed to add previously unseen buildings and to improve the geometric accuracy of the existing 3D building models. In this way, the system gradually accumulates evidence over time to make the site model more complete and more accurate. An extensive performance evaluation of component algorithms and the full system has been carried out. Two-dimensional building detection accuracy, as well as accuracy of the three-dimensional building reconstruction, are presented for a representative data set. <s> BIB008
Shadows  BIB001   and Walls (Vertical Edges)   BIB001  ] are very good evidence for the 3D-interpretation of mono images. Nevertheless, in some cases there are problems: For shadows they emerge from non-planar terrain close to the building, from shadows cast on buildings close-by, or from the fact that shadows could be occluded by the object itself in oblique views. On the other hand, vertical walls are mainly visible in oblique views but can be occluded by other objects. Nevertheless, it is not clear, why shadows, and especially vertical edges are used so seldom for building extraction from two or more images. The 3D-Geometry in  Images  BIB006 BIB007 BIB008 BIB004 BIB002 BIB005  based on a camera model and given orientations gives a valuable indication for the existence of the 3D-structure characteristic for buildings. The more images from different directions are used, the higher is the chance to exclude wrong matches. A Generic 3D-Model  which consists of surfaces and a Constructive Solid Geometry (CSG) modeling is maybe the best starting point for a more generic building extraction. All other representations either cannot describe a complex building properly, or it seems difficult to decide that a structure cannot be a building BIB006 . As a restrictive comment it has to be added that it is not yet clear if there are structures which "cannot be a building". Aspects BIB003  derived from a generic description consisting of building terminals and connectors allow for a parallel modeling in 2D image and 3D object model by enabling an explicit transition between these two. Automatic Object Extraction from Aerial Imagery 