SafeAPT: Safe Simulation-to-Real Robot Learning Using Diverse Policies Learned in Simulation

Rituraj Kaushik*, Karol Arndt, Ville Kyrki

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

4 Citations (Scopus)
21 Downloads (Pure)


The framework of sim-to-real learning, i.e., training policies in simulation and transferring them to real-world systems, is one of the most promising approaches towards data-efficient learning in robotics. However, due to the inevitable reality gap between the simulation and the real world, a policy learned in the simulation may not always generate a safe behaviour on the real robot. As a result, during policy adaptation in the real world, the robot may damage itself or cause harm to its surroundings. In this work, we introduce SafeAPT, a multi-goal robot learning algorithm that leverages a diverse repertoire of policies evolved in simulation and transfers the most promising safe policy to the real robot through episodic interaction. To achieve this, SafeAPT iteratively learns probabilistic reward and safety models from real-world observations using simulated experiences as priors. Then, it performs Bayesian optimization to select the best policy from the repertoire with the reward model, while maintaining the specified safety constraint using the safety model. SafeAPT allows a robot to adapt to a wide range of goals safely with the same repertoire of policies evolved in the simulation. We compare SafeAPT with several baselines, both in simulated and real robotic experiments, and show that SafeAPT finds high-performing policies within a few minutes of real-world operation while minimizing safety violations during the interactions.

Original languageEnglish
Pages (from-to)6838-6845
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number3
Publication statusPublished - 1 Jul 2022
MoE publication typeA1 Journal article-refereed


  • Evolutionary robotics
  • learning from experience
  • machine learning for robot control


Dive into the research topics of 'SafeAPT: Safe Simulation-to-Real Robot Learning Using Diverse Policies Learned in Simulation'. Together they form a unique fingerprint.

Cite this