Transferring Generalizable Motor Primitives From Simulation to Real World

Murtaza Hazara*, Ville Kyrki

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

155 Downloads (Pure)

Abstract

Reinforcement learning provides robots with an autonomous learning framework where a skill can he learned by exploration. Exploration in real world is, however, inherently unsafe and time consuming, and causes wear and tear. To address these, learning policies in simulation and then transferring them to physical systems has been proposed. In this letter, we propose a novel sample-efficient transfer approach, which is agnostic to the dynamics of a simulated system and combines it with incremental learning. Instead of transferring a single control policy, we transfer a generalizable contextual policy generated in simulation using one or few samples from real world to a target global model, which can generate policies across parameterized real-world situations. We studied the generalization capability of the incremental transfer framework using MuJoCo physics engine and KUKA LBR 4+. Experiments with ball-in-a-cup and basketball tasks demonstrated that the target model improved the generalization capability beyond the direct use of the source model indicating the effectiveness of the proposed framework. Experiments also indicated that the transfer capability depends on the generalization capability of the corresponding source model, similarity between source and target environment, and number of samples used for transferring.

Original languageEnglish
Pages (from-to)2172-2179
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume4
Issue number2
DOIs
Publication statusPublished - Apr 2019
MoE publication typeA1 Journal article-refereed

Keywords

  • Learning and Adaptive Systems
  • Model Learning for Control
  • Task analysis
  • Context modeling
  • Databases
  • Uncertainty
  • Robots
  • Friction

Cite this