This paper describes a software toolbox for the qualitative and quantitative evaluation of LiDAR (Light Detection and Ranging) and LiDAR-Inertial Odometry. We provide software as https://github.com/MapsHD organization with all necessary information at https://github.com/MapsHD/HDMapping. Our software contributions include a) a ground truth data processing tool, b) Dockerized state-of-the-art LO and LIO algorithms, c) multi-session data registration to a common coordinate system, d) Absolute Pose Error (APE) and Relative Pose Error (RPE) metrics, e) import/export tools for state-of-the-art LO and LIO algorithms, f) multi-session data registration to a common coordinate system, g) Absolute Pose Error (APE) and Relative Pose Error (RPE) metrics-of-the-art LO and LIO algorithms, c) multi-session data registration to a common coordinate system, d) Absolute Pose Error (APE) and Relative Pose Error (RPE) metrics, e) import/export tools for easier 3D data handling and visualization, e.g., in Cloud Compare software. This software is compatible with ROS1 (Robot Operating System) and ROS2 data formats. We present an example benchmark of the LeGO-LOAM, LIO-SAM, FAST-LIO, DLO, VoxelMap, Faster-LIO, KISS-ICP, CT-ICP, SLICT, DLIO, GLIM, iG-LIO, LIO-EKF, I2EKF-LO, GenZ-ICP, RESPLE, odometryroswrapper, Point-LIO, and LOAM-Livox algorithms. We provide videos for all experiments. The contribution of this paper is a software-oriented LO/LIO algorithm benchmark suite. The novelty lies in the integration of multiple benchmarking steps into a unified framework, thereby reducing the overall effort required for qualitative and quantitative evaluation.
Authors: Janusz Będkowski, Michał Pełka, Karol Majek, Marcin Matecki, Adrian Radulescu, Charles Hamesse, Ethan Decleyn, Przemysław Lekston, Tomasz Owerko, Przemysław Kuras, Michał Ciszewski, Jakub Kolecki, Karolina Tomaszkiewicz, Łukasz Ambroziński, Joanna Koszyk, Bartosz Hyla, Karolina Pargieła, Anna Malczewska, Tomasz Lipecki, Artur Adamek, Bartosz Mitka, Przemysław Klapa, Pelagia Gawronek, Martin Mokros, Jozef Výboštok, Juliána Chudá, Michal Skladan, Carlos Cabo, Kim André Anstensen, Daniel-Marian Craciun, Antun Jakopec, Michal Wlasiuk, Kornel Mrozowski, Maksymilian Kulicki, Krzysztof Stereńczak, Oskar Bartosz, Jakub Markiewicz, Sławomir Łapiński, Adam Kostrzewa, Mariana Campos, Machi Zawidzki, Jacek Szklarski, Rami Faraj, Loris Redovniković, Jurica Jagetić, Samer Karam, Răzvan Dumbravă, Milosz Mielcarek, Grzegorz Krok, Michal Laszkowski, Jaroslaw Wajs, Jakub Chudziński