28/08/2018 ROS, SLAM, Metrics, evaluation, SSIM, MSE, NE

Evaluation of grid maps produced by a SLAM algorithm

When evaluating the performance of a SLAM algorithm, quantifying the produced map quality is one of the most important criteria. Often, the produced map is compared with (1) a ground-truth map (which can be easily obtained in simulation) or (2) with another existing map that is considered accurate (in case of real world experiment where the ground-truth is not always available ).

Basically, grid maps are images, so image similarity measurement metrics can be used in this case. In this post, we consider three different metrics: Mean Square Error (MSE), K-nearest based normalized error (NE) and Structure Similarity Index (SSIM)

Setting up a software stack for autonomous mono robot exploration and mapping using ROS


This setup is performed and documented on an Ubuntu system with the following software stack:

  • Ubuntu 16.04 LTS (may work on other distribution though)
  • ROS kinetic: if you don't have ROS pre-installed, please refer to this tutorial http://wiki.ros.org/kinetic/Installation/Ubuntu. I suggest to use the full desktop installation configuration, this will take a while (> 2 GB)
  • Gazebo for simulation (it is installed by default if you choose the full desktop installation when installing ROS)
  • GIT (sudo apt-get install git)
  • Catkin for package building (installed by default when installing ROS)

To follow this post, some basic knowledge on ROS is needed:

Powered by antd server, (c) 2017 - 2018 Xuan Sang LE