This paper presents a new approach for integrating semantic information for vision-based vehicle navigation. Although vision-based vehicle navigation systems using pre-mapped visual landmarks are capable of achieving submeter level accuracy in large-scale urban environment, a typical error source in this type of systems comes from the presence of visual landmarks or features from temporal objects in the environment, such as cars and pedestrians. We propose a gated factor graph framework to use semantic information associated with visual features to make decisions on outlier inlier computation from three perspectives: the feature tracking process, the geo-referenced map building process, and the navigation system using pre-mapped landmarks. The class category that the visual feature belongs to is extracted from a pre-trained deep learning network trained for semantic segmentation. The feasibility and generality of our approach is demonstrated by our implementations on top of two vision-based navigation systems. Experimental evaluations validate that the injection of semantic information associated with visual landmarks using our approach achieves substantial improvements in accuracy on GPS-denied navigation solutions for large-scale urban scenarios
Note the main interest of this work is the overall global navigation accuracy including places where only few or no valid visual landmarks are available due to scene occlusion or appearance change. Thus, we evaluate our GPS-denied navigation accuracy against ground truths provided by fusing high-precision differential GPS with IMUs, which is different from calculating the localization error as the relative distance to the visual map @cite_15 .
@cite_15
2029841124 2115555252 2538522345 2160626949
Next generation driver assistance systems require precise self localization. Common approaches using global navigation satellite systems (GNSSs) suffer from multipath and shadowing effects often rendering this solution insufficient. In urban environments this problem becomes even more pronounced. Herein we present a system for six degrees of freedom (DOF) ego localization using a mono camera and an inertial measurement unit (IMU). The camera image is processed to yield a rough position estimate using a previously computed landmark map. Thereafter IMU measurements are fused with the position estimate for a refined localization update. Moreover, we present the mapping pipeline required for the creation of landmark maps. Finally, we present experiments on real world data. The accuracy of the system is evaluated by computing two independent ego positions of the same trajectory from two distinct cameras and investigating these estimates for consistency. A mean localization accuracy of 10 cm is achieved on a 10 km sequence in an inner city scenario. This paper presents results demonstrating real-time six degree of freedom localization, mapping, navigation and obstacle avoidance in an outdoor environment using only a lowcost o-the-shelf inertial measurement unit and a monocular camera. This navigation system is intended for operation of small unmanned aerial vehicles when GPS signals are unavailable due to obstructions or jamming and when operating in cluttered environments such as urban canyons or forests. A small radio-controlled car is used as a test bed. A bearings-only Simultaneous Localization and Mapping algorithm was implemented to localize both the vehicle and obstacles. This paper describes: (a) the hardware used; (b) a two-step approach for data association (image frame to image frame followed by image frame to map); (c) a technique for landmark initialization for the case where landmarks are located on the ground. Hardware test results demonstrating navigation to a goal in an obstacle strewn environment are presented. The eect of unmodelled sensor biases is examined in simulation. In recent years there have been excellent results in visual-inertial odometry techniques, which aim to compute the incremental motion of the sensor with high accuracy and robustness. However, these approaches lack the capability to close loops and trajectory estimation accumulates drift even if the sensor is continually revisiting the same place. In this letter, we present a novel tightly coupled visual-inertial simultaneous localization and mapping system that is able to close loops and reuse its map to achieve zero-drift localization in already mapped areas. While our approach can be applied to any camera configuration, we address here the most general problem of a monocular camera, with its well-known scale ambiguity. We also propose a novel IMU initialization method, which computes the scale, the gravity direction, the velocity, and gyroscope and accelerometer biases, in a few seconds with high accuracy. We test our system in the 11 sequences of a recent micro-aerial vehicle public dataset achieving a typical scale factor error of 1 and centimeter precision. We compare to the state-of-the-art in visual-inertial odometry in sequences with revisiting, proving the better accuracy of our method due to map reuse and no drift accumulation. This paper presents a new method for improving the accuracy of inertial measurement units (IMUs) mounted on land vehicles. The algorithm exploits nonholonomic constraints that govern the motion of a vehicle on a surface to obtain velocity observation measurements which aid in the estimation of the alignment of the IMU as well as the forward velocity of the vehicle. It is shown that this can be achieved without any external sensing provided that certain observability conditions are met. A theoretical analysis is provided together with a comparison of experimental results between a nonlinear implementation of the algorithm and an IMU GPS navigation system. This comparison demonstrates the effectiveness of the algorithm. The real time implementation is also addressed through a multiple observation inertial aiding algorithm based on the information filter. The results show that the use of these constraints and vehicle speed guarantees the observability of the velocity and the attitude of the inertial unit, and hence bounds the errors associated with these states. The strategies proposed provides a tighter navigation loop which can sustain outages of GPS for a greater amount of time as compared to when the inertial unit is used with standard integration algorithms.