Today's autonomous vehicles rely extensively on high-definition 3D maps to navigate the environment. While this approach works well when these maps are completely up-to-date, safe autonomous vehicles must be able to corroborate the map's information via a real time sensor-based system. Our goal in this work is to develop a model for road layout inference given imagery from on-board cameras, without any reliance on high-definition maps. However, no sufficient dataset for training such a model exists. Here, we leverage the availability of standard navigation maps and corresponding street view images to construct an automatically labeled, large-scale dataset for this complex scene understanding problem. By matching road vectors and metadata from navigation maps with Google Street View images, we can assign ground truth road layout attributes (e.g., distance to an intersection, one-way vs. two-way street) to the images. We then train deep convolutional networks to predict these road layout attributes given a single monocular RGB image. Experimental evaluation demonstrates that our model learns to correctly infer the road attributes using only panoramas captured by car-mounted cameras as input. Additionally, our results indicate that this method may be suitable to the novel application of recommending safety improvements to infrastructure (e.g., suggesting an alternative speed limit for a street).
Recently, researchers have exploited the OSM or GSV datasets for a few localization-related applications. @cite_3 paired visual odometry with OSM to alleviate visual odometry's tendency to drift. In @cite_12 , models estimating road attributes including intersection presence and road type (trained with OSM-extracted labels) were used to speed up self-localization on driving video. In @cite_11 , M ' developed a model to segment OSM roads in aerial images, thereby providing road widths and enhancing the map's accuracy and applicability to localization. @cite_13 trained a ConvNet-based model to match GSV images with their corresponding location in aerial images. While we construct and leverage a dataset based on OSM and GSV, our model does not attempt to localize the street view image and instead estimates road layout attributes given the surrounding visual scene.
@cite_13 @cite_12 @cite_3 @cite_11
2205800717 2070708339 2766577666 2023671107
In recent years, contextual models that exploit maps have been shown to be very effective for many recognition and localization tasks. In this paper we propose to exploit aerial images in order to enhance freely available world maps. Towards this goal, we make use of OpenStreetMap and formulate the problem as the one of inference in a Markov random field parameterized in terms of the location of the road-segment centerlines as well as their width. This parameterization enables very efficient inference and returns only topologically correct roads. In particular, we can segment all OSM roads in the whole world in a single day using a small cluster of 10 computers. Importantly, our approach generalizes very well, it can be trained using only 1.5 km2 aerial imagery and produce very accurate results in any location across the globe. We demonstrate the effectiveness of our approach outperforming the state-of-the-art in two new benchmarks that we collect. We then show how our enhanced maps are beneficial for semantic segmentation of ground images. In order to support driver assistance systems in unconstrained environments, we propose to extend local appearance-based road classification with a spatial feature generation and classification. Therefore, a hierarchical approach consisting of multiple low level base classifiers, the novel spatial feature generation, as well as a final road terrain classification, is used. The system perceives a variety of local properties of the environment by means of base classifiers operating on patches extracted from monocular camera images, each represented in a metric confidence map. The core of the proposed approach is the computation of spatial ray features (SPRAY) from these confidence maps. With this, the road-terrain classifier can decide based on local visual properties and their spatial layout in the scene. In order to show the feasibility of the approach, the extraction and evaluation of the metric ego-lane driving corridor on an inner city stream is demonstrated. This is a challenging task because on a local appearance level, ego-lane is not distinguishable from other asphalt parts on the road. However, by incorporating the proposed SPRAY features the distinction is possible without requiring an explicit lane model. Due to the parallel structure of this bottom-up approach, the implemented system operates in real-time with approximately 25 Hz on a GPU. In this paper, we address semantic segmentation of road-objects from 3D LiDAR point clouds. In particular, we wish to detect and categorize instances of interest, such as cars, pedestrians and cyclists. We formulate this problem as a point- wise classification problem, and propose an end-to-end pipeline called SqueezeSeg based on convolutional neural networks (CNN): the CNN takes a transformed LiDAR point cloud as input and directly outputs a point-wise label map, which is then refined by a conditional random field (CRF) implemented as a recurrent layer. Instance-level labels are then obtained by conventional clustering algorithms. Our CNN model is trained on LiDAR point clouds from the KITTI dataset, and our point-wise segmentation labels are derived from 3D bounding boxes from KITTI. To obtain extra training data, we built a LiDAR simulator into Grand Theft Auto V (GTA-V), a popular video game, to synthesize large amounts of realistic training data. Our experiments show that SqueezeSeg achieves high accuracy with astonishingly fast and stable runtime (8.7 ms per frame), highly desirable for autonomous driving applications. Furthermore, additionally training on synthesized data boosts validation accuracy on real-world data. Our source code and synthesized data will be open-sourced. This paper proposes a method for achieving improved ego-vehicle global localization with respect to an approaching intersection, which is based on the alignment of visual landmarks perceived by the on-board visual system, with the information from a proposed extended digital map (EDM). The visual system relies on a stereovision system that provides a detailed 3-D description of the environment, including road landmark information (lateral lane delimiters, painted traffic signs, curbs, and stop lines) and dynamic environment information (other vehicles). An EDM is proposed, which enriches the standard map information with a detailed description of the intersection required for current lane identification, landmark alignment, and ego-vehicle accurate global localization. A novel approach for lane-delimiter classification, which is necessary for the lane identification, is also presented. An original solution for identifying the current lane, combining visual and map information with the help of a Bayesian network (BN), is proposed. Extensive experiments have been performed, and the results are evaluated with a Global Navigation Satellite System of high accuracy (2 cm). The achieved global localization accuracy is of submeter level, depending on the performance of the stereovision system.