SafeAPT: Safe Simulation-to-Real Robot Learning Using Diverse Policies Learned in Simulation

Rituraj Kaushik*, Karol Arndt, Ville Kyrki

*Tämän työn vastaava kirjoittaja

Tutkimustuotos: LehtiartikkeliArticleScientificvertaisarvioitu

11 Lataukset (Pure)

Abstrakti

The framework of sim-to-real learning, i.e., training policies in simulation and transferring them to real-world systems, is one of the most promising approaches towards data-efficient learning in robotics. However, due to the inevitable reality gap between the simulation and the real world, a policy learned in the simulation may not always generate a safe behaviour on the real robot. As a result, during policy adaptation in the real world, the robot may damage itself or cause harm to its surroundings. In this work, we introduce SafeAPT, a multi-goal robot learning algorithm that leverages a diverse repertoire of policies evolved in simulation and transfers the most promising safe policy to the real robot through episodic interaction. To achieve this, SafeAPT iteratively learns probabilistic reward and safety models from real-world observations using simulated experiences as priors. Then, it performs Bayesian optimization to select the best policy from the repertoire with the reward model, while maintaining the specified safety constraint using the safety model. SafeAPT allows a robot to adapt to a wide range of goals safely with the same repertoire of policies evolved in the simulation. We compare SafeAPT with several baselines, both in simulated and real robotic experiments, and show that SafeAPT finds high-performing policies within a few minutes of real-world operation while minimizing safety violations during the interactions.

AlkuperäiskieliEnglanti
Sivut6838-6845
Sivumäärä8
JulkaisuIEEE Robotics and Automation Letters
Vuosikerta7
Numero3
DOI - pysyväislinkit
TilaJulkaistu - 1 heinäk. 2022
OKM-julkaisutyyppiA1 Julkaistu artikkeli, soviteltu

Sormenjälki

Sukella tutkimusaiheisiin 'SafeAPT: Safe Simulation-to-Real Robot Learning Using Diverse Policies Learned in Simulation'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.
  • FCAI: Suomen tekoälykeskus

    Kaski, S.

    01/01/201931/12/2022

    Projekti: Academy of Finland: Other research funding

Siteeraa tätä