Bundle adjustment (BA) is a fundamental optimization technique used in many crucial applications, including 3D scene reconstruction, robotic localization, camera calibration, autonomous driving, space exploration, street view map generation etc. Essentially, BA is a joint non-linear optimization problem, and one which can consume a significant amount of time and power, especially for large optimization problems. Previous approaches of optimizing BA performance heavily rely on parallel processing or distributed computing, which trade higher power consumption for higher performance. In this paper we propose -BA, the first hardware-software co-designed BA engine on an embedded FPGA-SoC that exploits custom hardware for higher performance and power efficiency. Specifically, based on our key observation that not all points appear on all images in a BA problem, we designed and implemented a Co-Observation Optimization technique to accelerate BA operations with optimized usage of memory and computation resources. Experimental results confirm that -BA outperforms the existing software implementations in terms of performance and power consumption.
Parallel processing using multicore, either on CPU or GPU, can be applied to optimize BA performance. Wu presented multicore solutions to the problem of bundle adjustment that run on currently available CPUs and GPUs @cite_1 . The authors concluded that using multicore systems deliver a 10x to 30x boost in speed over existing systems while reducing the amount of memory used. This was achieved by carefully restructuring the matrix vector product used in the PCG iterations into easily parallelizable operations. This restructuring also opens the door to a matrix free implementation which leads to substantial reductions in the memory consumption as well as execution time. The authors also showed that single precision arithmetic when combined with appropriate normalization gives numerical performance comparable to double precision based solvers while further reducing the memory and time cost. The resulting system enabled running the largest bundle adjustment problems to date on a single GPU.
@cite_1
2001790138
We present the design and implementation of new inexact Newton type Bundle Adjustment algorithms that exploit hardware parallelism for efficiently solving large scale 3D scene reconstruction problems. We explore the use of multicore CPU as well as multicore GPUs for this purpose. We show that overcoming the severe memory and bandwidth limitations of current generation GPUs not only leads to more space efficient algorithms, but also to surprising savings in runtime. Our CPU based system is up to ten times and our GPU based system is up to thirty times faster than the current state of the art methods [1], while maintaining comparable convergence behavior. The code and additional results are available at http: grail.cs. washington.edu projects mcba.