This paper describes a software toolbox for LiDAR (Light Detection and Ranging) and LiDAR-Inertial Odometry qualitative and quantitative evaluation. We provide software as https://github.com/MapsHD organization with all necessary information at https://github.com/MapsHD/HDMapping. Our software contributions are a) ground truth data processing tool, b) dockerized state-of-the-art LO and LIO algorithms, c) multi-session data registration to common coordinate system, d) Absolute Pose Error (APE) and Relative Pose Error (RPE) metrics, e) import/export tools for easier 3D data handling and visualizing, e.g., in Cloud Compare software. This software is compatible with ROS1 (Robot Operating System) and ROS2 data formats. We show an example benchmark of LeGO-LOAM, LIO-SAM, FAST-LIO, DLO, VoxelMap, Faster-LIO, KISS-ICP, CT-ICP, SLICT, DLIO, GLIM, iG-LIO, LIO-EKF, I2EKF-LO, GenZ-ICP, RESPLE, odometryroswrapper, Point-LIO, and LOAM-Livox algorithms. For all experiments we provide movies. The contribution of the paper is software-oriented LO/LIO algorithm benchmark suite. The novelty lies in the integration of multiple benchmarking steps into a unified framework, thus overall effort needed for qualitative and quantitative evaluation is reduced.
Autorzy: Janusz Bȩdkowski , Michał Pełka, Karol Majek , Marcin Matecki , Adrian Radulescu , Charles Hamesse , Ethan Decleyn , Przemysław Lekston , Tomasz Owerko , Przemysław Kuras , Michał Ciszewski , Jakub Kolecki , Karolina Tomaszkiewicz , Łukasz Ambroziński , Joanna Koszyk , Bartosz Hyla , Karolina Pargieła , Anna Malczewska , Tomasz Lipecki , Artur Adamek , Bartosz Mitka , Klapa Przemysław , Pelagia Gawronek , Martin Mokros , Jozef Výboštok , Juliána Chudá , Michal Skladan , Carlos Cabo , Kim André Anstensen , Daniel-Marian Craciun , Antun Jakopec , Michal Wlasiuk , Kornel Mrozowski , Maksymilian Kulicki , Krzysztof Stereńczak , Oskar Bartosz , Jakub Markiewicz , Sławomir Łapiński , Adam Kostrzewa , Mariana Campos , Machi Zawidzki , Jacek Szklarski , Rami Faraj , Loris Redovniković , Jurica Jagetić , Samer Karam , Răzvan Dumbravă , Milosz Mielcarek , Grzegorz Krok , Michal Laszkowski , Jaroslaw Wajs , Jakub Chudziński