We introduce a novel strategy for learning to extract semantically meaningful features from aerial imagery. Instead of manually labeling the aerial imagery, we propose to predict (noisy) semantic features automatically extracted from co-located ground imagery. Our network architecture takes an aerial image as input, extracts features using a convolutional neural network, and then applies an adaptive transformation to map these features into the ground-level perspective. We use an end-to-end learning approach to minimize the difference between the semantic segmentation extracted directly from the ground image and the semantic segmentation predicted solely based on the aerial image. We show that a model learned using this strategy, with no additional training, is already capable of rough semantic labeling of aerial imagery. Furthermore, we demonstrate that by finetuning this model we can achieve more accurate semantic segmentation than two baseline initialization strategies. We use our network to address the task of estimating the geolocation and geoorientation of a ground image. Finally, we show how features extracted from an aerial image can be used to hallucinate a plausible ground-level panorama.
There is a long tradition of using computer vision techniques for aerial and satellite image understanding @cite_18 @cite_16 @cite_19 . Historically these two domains were distinct. Satellite imagery was typically lower-resolution, from a strictly top-down view, and with a diversity of spectral bands. Aerial imagery was typically higher-resolution, with a greater diversity of viewing angles, but with only RGB and NIR sensors. Recently these two domains have converged; we will use the term aerial imagery as we are primarily working with high-resolution RGB imagery. However, our approach could be applied to many types of aerial and satellite imagery. Kluckner al @cite_13 address the task of semantic segmentation using a random forest to combine color and height information. More recent work has explored the use of CNNs for aerial image understanding. Mnih and Hinton propose a CNN for detecting roads in aerial imagery @cite_20 using GIS data as ground truth. They extend their approach to handle omission noise and misregistration between the imagery and the labels @cite_28 . These approaches require either extensive pixel-level manual annotation or existing GIS data. Our work is the first to demonstrate the ability to transfer a dense pixel-level labeling of ground imagery to aerial imagery.
@cite_18 @cite_28 @cite_19 @cite_16 @cite_13 @cite_20
2155910279 2252268321 2308318555 2102792793 73112891
We introduce a novel strategy for learning to extract semantically meaningful features from aerial imagery. Instead of manually labeling the aerial imagery, we propose to predict (noisy) semantic features automatically extracted from co-located ground imagery. Our network architecture takes an aerial image as input, extracts features using a convolutional neural network, and then applies an adaptive transformation to map these features into the ground-level perspective. We use an end-to-end learning approach to minimize the difference between the semantic segmentation extracted directly from the ground image and the semantic segmentation predicted solely based on the aerial image. We show that a model learned using this strategy, with no additional training, is already capable of rough semantic labeling of aerial imagery. Furthermore, we demonstrate that by finetuning this model we can achieve more accurate semantic segmentation than two baseline initialization strategies. We use our network to address the task of estimating the geolocation and geo-orientation of a ground image. Finally, we show how features extracted from an aerial image can be used to hallucinate a plausible ground-level panorama. The recent availability of geo-tagged images and rich geospatial data has inspired a number of algorithms for image based geolocalization. Most approaches predict the location of a query image by matching to ground-level images with known locations (e.g., street-view data). However, most of the Earth does not have ground-level reference photos available. Fortunately, more complete coverage is provided by oblique aerial or bird's eye imagery. In this work, we localize a ground-level query image by matching it to a reference database of aerial imagery. We use publicly available data to build a dataset of 78K aligned crossview image pairs. The primary challenge for this task is that traditional computer vision approaches cannot handle the wide baseline and appearance variation of these cross-view pairs. We use our dataset to learn a feature representation in which matching views are near one another and mismatched views are far apart. Our proposed approach, Where-CNN, is inspired by deep learning success in face verification and achieves significant improvements over traditional hand-crafted features and existing deep features learned from other large-scale databases. We show the effectiveness of Where-CNN in finding matches between street view and aerial view imagery and demonstrate the ability of our learned features to generalize to novel locations. The increased availability of high resolution satellite imagery allows to sense very detailed structures on the surface of our planet. Access to such information opens up new directions in the analysis of remote sensing imagery. However, at the same time this raises a set of new challenges for existing pixel-based prediction methods, such as semantic segmentation approaches. While deep neural networks have achieved significant advances in the semantic segmentation of high resolution images in the past, most of the existing approaches tend to produce predictions with poor boundaries. In this paper, we address the problem of preserving semantic segmentation boundaries in high resolution satellite imagery by introducing a new cascaded multi-task loss. We evaluate our approach on Inria Aerial Image Labeling Dataset which contains large-scale and high resolution images. Our results show that we are able to outperform state-of-the-art methods by 8.3 without any additional post-processing step. We are witnessing daily acquisition of large amounts of aerial and satellite imagery. Analysis of such large quantities of data can be helpful for many practical applications. In this letter, we present an automatic content-based analysis of aerial imagery in order to detect and mark arbitrary objects or regions in high-resolution images. For that purpose, we proposed a method for automatic object detection based on a convolutional neural network. A novel two-stage approach for network training is implemented and verified in the tasks of aerial image classification and object detection. First, we tested the proposed training approach using UCMerced data set of aerial images and achieved accuracy of approximately 98.6 . Second, the method for automatic object detection was implemented and verified. For implementation on GPGPU, a required processing time for one aerial image of size 5000 @math 5000 pixels was around 30 s. This paper describes a method of combined ultra-high resolution satellite imaging, unmanned aerial vehicle (UAV) photography, and sub-surface geophysical investigation for anomaly detection, which was employed in a non-invasive survey of three archaeological sites in Northern Mongolia. The surveyed sites were a Bronze Age burial mound, a Turkish period tomb, and a steppe city fortification of unknown origin. For the satellite survey, 50 cm resolution pan-sharpened imagery was generated through a combination of multispectral and panchromatic data, collected from the GeoEye-1 earth-sensing satellite. The imagery was then used to identify the location of the aforementioned sites in an approximate area of 3000 km2 . Aerial photographs of the sites were obtained with two customized electric-powered UAVs: a fixed flying wing rear-propulsion plane and a multi-propeller oktokopter helicopter system. Finally, geophysical investigation was conducted with a GSM-19 Overhouser gradiometer, an EM38 electromagnetometer, and an IDS Detector Duo ground penetrating radar. The satellite imagery and aerial photographs were combined with the geophysical survey results and on-site surface observations to provide insight and contextual information about anomalies in multiple layers of data. The results highlight the effectiveness and robustness of the employed method for archaeological investigation in large, rugged and scarcely populated areas. Automation of objects labeling in aerial imagery is a computer vision task with numerous practical applications. Fields like energy exploration require an automated method to process a continuous stream of imagery on a daily basis. In this paper we propose a pipeline to tackle this problem using a stack of convolutional neural networks (U-Net architecture) arranged end-to-end. Each network works as post-processor to the previous one. Our model outperforms current state-of-the-art on two different datasets: Inria Aerial Image Labeling dataset and Massachusetts Buildings dataset each with different characteristics such as spatial resolution, object shapes and scales. Moreover, we experimentally validate computation time savings by processing sub-sampled images and later upsampling pixelwise labeling. These savings come at a negligible degradation in segmentation quality. Though the conducted experiments in this paper cover only aerial imagery, the technique presented is general and can handle other types of images.