For human drivers, having rear and side-view mirrors is vital for safe driving. They deliver a more complete view of what is happening around the car. Human drivers also heavily exploit their mental map for navigation. Nonetheless, several methods have been published that learn driving models with only a front-facing camera and without a route planner. This lack of information renders the self-driving task quite intractable. We investigate the problem in a more realistic setting, which consists of a surround-view camera system with eight cameras, a route planner, and a CAN bus reader. In particular, we develop a sensor setup that provides data for a 360-degree view of the area surrounding the vehicle, the driving route to the destination, and low-level driving maneuvers (e.g. steering angle and speed) by human drivers. With such a sensor setup we collect a new driving dataset, covering diverse driving scenarios and varying weather illumination conditions. Finally, we learn a novel driving model by integrating information from the surround-view cameras and the route planner. Two route planners are exploited: 1) by representing the planned routes on OpenStreetMap as a stack of GPS coordinates, and 2) by rendering the planned routes on TomTom Go Mobile and recording the progression into a video. Our experiments show that: 1) 360-degree surround-view cameras help avoid failures made with a single front-view camera, in particular for city driving and intersection scenarios; and 2) route planners help the driving task significantly, especially for steering angle prediction.
While considerable progress has been made both in computer vision and in route planning, their integration for learning driving models has not received due attention in the academic community. A trending topic is to combine digital maps and street-view images for accurate vehicle localization @cite_24 @cite_71 @cite_6 @cite_23 .
@cite_24 @cite_23 @cite_6 @cite_71
1985434019 2771230335 2023671107 1969553439
Accurate, reliable, and affordable vehicle localization is one important task in current automotive research activities. It enables technologies like cooperative systems or enhanced map based assistance systems. There are a wide variety of approaches to reach this higher accuracy. The algorithm presented in this paper utilizes image landmarks in combination with a low-cost Global Navigation Satellite System (GNSS) receiver and vehicle odometry to achieve this. While similar approaches often extract features from camera images and match those features with map information, the algorithm presented in this work directly transforms map feature data, creating a image of map features, like the camera would see it. The evaluation of this image prediction uses the coherency value, which is derived from the structure tensor. By predicting the whole image, the incorporation of the map information is moved from feature level to signal level. The likelihood models used for the evaluation of the coherency image are derived from real, manually labeled data. We present promising results of a test drive in an area with complex intersections. Those results are compared to ground truth data. Maps are a key component in image-based camera localization and visual SLAM systems: they are used to establish geometric constraints between images, correct drift in relative pose estimation, and relocalize cameras after lost tracking. The exact definitions of maps, however, are often application-specific and hand-crafted for different scenarios (e.g., 3D landmarks, lines, planes, bags of visual words). We propose to represent maps as a deep neural net called MapNet, which enables learning a data-driven map representation. Unlike prior work on learning maps, MapNet exploits cheap and ubiquitous sensory inputs like visual odometry and GPS in addition to images and fuses them together for camera localization. Geometric constraints expressed by these inputs, which have traditionally been used in bundle adjustment or pose-graph optimization, are formulated as loss terms in MapNet training and also used during inference. In addition to directly improving localization accuracy, this allows us to update the MapNet (i.e., maps) in a self-supervised manner using additional unlabeled video sequences from the scene. We also propose a novel parameterization for camera rotation which is better suited for deep-learning based camera pose regression. Experimental results on both the indoor 7-Scenes dataset and the outdoor Oxford RobotCar dataset show significant performance improvement over prior work. This paper proposes a method for achieving improved ego-vehicle global localization with respect to an approaching intersection, which is based on the alignment of visual landmarks perceived by the on-board visual system, with the information from a proposed extended digital map (EDM). The visual system relies on a stereovision system that provides a detailed 3-D description of the environment, including road landmark information (lateral lane delimiters, painted traffic signs, curbs, and stop lines) and dynamic environment information (other vehicles). An EDM is proposed, which enriches the standard map information with a detailed description of the intersection required for current lane identification, landmark alignment, and ego-vehicle accurate global localization. A novel approach for lane-delimiter classification, which is necessary for the lane identification, is also presented. An original solution for identifying the current lane, combining visual and map information with the help of a Bayesian network (BN), is proposed. Extensive experiments have been performed, and the results are evaluated with a Global Navigation Satellite System of high accuracy (2 cm). The achieved global localization accuracy is of submeter level, depending on the performance of the stereovision system. Estimating the pose in real-time is a primary function for intelligent vehicle navigation. Whilst different solutions exist, most of them rely on the use of high-end sensors. This paper proposes a solution that exploits an automotive type L1-GPS receiver, features extracted by low-cost perception sensors and vehicle proprioceptive information. A key idea is to use the lane detection function of a video camera to retrieve accurate lateral and orientation information with respect to road lane markings. To this end, lane markings are mobile-mapped by the vehicle itself during a first stage by using an accurate localizer. Then, the resulting map allows for the exploitation of camera-detected features for autonomous real-time localization. The results are then combined with GPS estimates and dead-reckoning sensors in order to provide localization information with high availability. As L1-GPS errors can be large and are time correlated, we study in the paper several GPS error models that are experimentally tested with shaping filters. The approach demonstrates that the use of low-cost sensors with adequate data-fusion algorithms should lead to computer-controlled guidance functions in complex road networks.