Many tasks performed by autonomous vehicles such as road marking detection, object tracking, and path planning are simpler in bird's-eye view. Hence, Inverse Perspective Mapping (IPM) is often applied to remove the perspective effect from a vehicle's front-facing camera and to remap its images into a 2D domain, resulting in a top-down view. Unfortunately, however, this leads to unnatural blurring and stretching of objects at further distance, due to the resolution of the camera, limiting applicability. In this paper, we present an adversarial learning approach for generating a significantly improved IPM from a single camera image in real time. The generated bird'seye-view images contain sharper features (e.g, road markings) and a more homogeneous illumination, while (dynamic) objects are automatically removed from the scene, thus revealing the underlying road layout in an improved fashion. We demonstrate our framework using real-world data from the Oxford Robot-Car Dataset and show that scene understanding tasks directly benefit from our boosted IPM approach.
Very recently, @cite_12 proposed the first learning approach for IPM using a synthetic dataset. The authors introduced BridgeGAN which employs the homography IPM to bridge the significant appearance gap between the frontal view and bird's-eye view. In contrast, we use real-world data and consequently labels to generate boosted IPM for larger scenes. Therefore, our learned mapping is directly beneficial for scene understanding tasks (see Section ).
@cite_12
2553321288 1484723123 2125878157 2768591600
This paper proposes an adaptive Inverse Perspective Mapping (IPM) algorithm to obtain accurate bird's-eye view images from the sequential images of forward looking cameras. These images are often distorted by the motion of the vehicle; even a small motion can cause a substantial effect on bird'seye view images. In this paper, we propose an adaptive model for the IPM to accurately transform camera images to bird'seye view images by using motion information. Using motion derived from the monocular visual simultaneous localization and mapping (SLAM), experimental result shows that the proposed approaches can provide stable bird's-eye view images, even with large motion during the drive. In this paper, the authors examine the issue of Inverse Perspective Mapping (IPM) which removes the perspective effect from acquired images that are associated with autonomous driving systems. They in turn present an Extended IPM (EIPM) which is based on stereo image processing and which is used to update the road slope ahead of the vehicle. The EIPM removes the assumption of a flat road ahead of the vehicle and allows for the recovery of road texture even in the presence of a slope. The authors describe how this technique is applied to synthetic images and how it has been integrated into the GOLD system on the ARGO autonomous vehicle. In this work, a new inverse perspective mapping (IPM) technique is proposed based on a robust estimation of the vanishing point, which provide bird-view images of the road, so that facilitating the tasks of road modeling and vehicle detection and tracking. This new approach has been design to cope with the instability that cameras mounted on a moving vehicle suffer. The estimation of the vanishing point relies on a novel and efficient feature extraction strategy, which segmentates the lane markings of the images by combining a histogram-based segmentation with temporal and frequency filtering. Then, the vanishing point of each image is stabilized by means of a temporal filtering along the estimates of previous images. In a last step, the IPM image is computed based on the stabilized vanishing point. Tests have been carried out on several long video sequences captured from cameras inside a vehicle being driven along highways and local roads, with different illumination and weather conditions, presence of shadows, occluding vehicles, and slope changes. Results have shown a significant improvement in terms of lane width constancy and parallelism between lane markings over non-stabilized IPM algorithms. In human learning, it is common to use multiple sources of information jointly. However, most existing feature learning approaches learn from only a single task. In this paper, we propose a novel multi-task deep network to learn generalizable high-level visual representations. Since multitask learning requires annotations for multiple properties of the same training instance, we look to synthetic images to train our network. To overcome the domain difference between real and synthetic data, we employ an unsupervised feature space domain adaptation method based on adversarial learning. Given an input synthetic RGB image, our network simultaneously predicts its surface normal, depth, and instance contour, while also minimizing the feature space domain differences between real and synthetic data. Through extensive experiments, we demonstrate that our network learns more transferable representations compared to single-task baselines. Our learned representation produces state-of-the-art transfer learning results on PASCAL VOC 2007 classification and 2012 detection.