Improving Repeatability of Experiments by Automatic Evaluation of SLAM Algorithms

The development of good experimental methodologies for robotics takes often inspiration from general principles of experimental practice. Repeatability prescribes that experiments should involve several trials in order to guarantee that results are not achieved by chance, but are systematic, and statistically significant trends can be identified. In this paper, we propose an approach to improve the repeatability of experiments performed in robotics. In particular, we focus on the domain of SLAM (Simultaneous Localization And Mapping) and we introduce a system that exploits simulations to generate a large number of test data on which SLAM algorithms are automatically evaluated in order to obtain consistent results, according to the principle of repeatability.