A Monte Carlo Framework for Incremental Improvement of Simulation Fidelity

Abstract

Robot software developed in simulation often does not behave as expected when deployed because the simulation does not sufficiently represent reality—this is sometimes called the ‘reality gap’ problem. We propose a novel algorithm to address the reality gap by injecting real-world experience into the simulation. It is assumed that the robot program (control policy) is developed using simulation, but subsequently deployed on a real system, and that the program includes a performance objective monitor procedure with scalar output. The proposed approach collects simulation and real world observations and builds conditional probability functions. These are used to generate paired roll-outs to identify points of divergence in simulation and real behavior. From these, state-space kernels are generated that, when integrated with the original simulation, coerce the simulation into behaving more like observed reality. Performance results are presented for a long-term deployment of an autonomous delivery vehicle example.

The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Department of Defense or the U.S. Government. Lyons was partially supported by DL-47359-15016 from Bloomberg LP.