Real-time dense computer vision and SLAM offer great potential for a new level of scene modelling, tracking and real environmental interaction for many types of robot, but their high computational requirements mean that use on mass market embedded platforms is challenging. Meanwhile, trends in low-cost, low-power processing are towards massive parallelism and heterogeneity, making it difficult for robotics and vision researchers to implement their algorithms in a performance-portable way. In this paper we introduce SLAMBench, a publicly-available software framework which represents a starting point for quantitative, comparable and validatable experimental research to investigate trade-offs in performance, accuracy and energy consumption of a dense RGB-D SLAM system. SLAMBench provides a KinectFusion implementation in C++, OpenMP, OpenCL and CUDA, and harnesses the ICL-NUIM dataset of synthetic RGB-D sequences with trajectory and scene ground truth for reliable accuracy comparison of different implementation and algorithms. We present an analysis and breakdown of the constituent algorithmic elements of KinectFusion, and experimentally investigate their execution time on a variety of multicore and GPU-accelerated platforms. For a popular embedded platform, we also present an analysis of energy efficiency for different configuration alternatives.
While such efforts are a step in the right direction, they do not provide the software tools for accuracy verification and exploitation of hardware accelerators or graphics processor units (GPUs). Nor do they enable investigation of energy consumption, performance and accuracy envelopes for 3D scene reconstruction algorithms across a range of hardware targets. The lack of benchmarks stems from the difficulty in systematically comparing the accuracy of the reconstruction while measuring the performance. In this work we focus specifically on SLAM and introduce a publicly-available framework for quantitative, comparable and validatable experimental research in the form of a benchmark for dense 3D scene understanding. A key feature of SLAMBench is that it is designed on top of the recently-proposed ICL-NUIM accuracy benchmark @cite_26 , and thus supports wider research in hardware and software. The quantitative evaluation of solution accuracy into SLAMBench enables algorithmic research to be performed. The latter is an important feature that is lacking in current performance benchmarks. A typical output of SLAMBench consists of the performance achieved, and the accuracy of the result along with the energy consumption (on platforms where such measurement is possible). These parameters capture the potential trade-offs for real-time vision platforms.
@cite_26
2058535340
We introduce the Imperial College London and National University of Ireland Maynooth (ICL-NUIM) dataset for the evaluation of visual odometry, 3D reconstruction and SLAM algorithms that typically use RGB-D data. We present a collection of handheld RGB-D camera sequences within synthetically generated environments. RGB-D sequences with perfect ground truth poses are provided as well as a ground truth surface model that enables a method of quantitatively evaluating the final map or surface reconstruction accuracy. Care has been taken to simulate typically observed real-world artefacts in the synthetic imagery by modelling sensor noise in both RGB and depth data. While this dataset is useful for the evaluation of visual odometry and SLAM trajectory estimation, our main focus is on providing a method to benchmark the surface reconstruction accuracy which to date has been missing in the RGB-D community despite the plethora of ground truth RGB-D datasets available.