Reinforcement learning for improving imitated in-contact skills

Murtaza Hazara, Ville Kyrki

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

5 Citations (Scopus)
297 Downloads (Pure)


Although motor primitives (MPs) for trajectory basedskills have been studied extensively, much less attentionhas been devoted to studying in-contact tasks. With robotsbecoming more commonplace, it is both economical andconvenient to have a mechanism for learning an in-contacttask from demonstration. However, transferring an in-contactskill such as wood planing from a human to a robot issignificantly more challenging than transferring a trajectory basedskill; it requires a simultaneous control of both poseand force. Furthermore, some in-contact tasks have extremelycomplex contact environments. We present a framework for imitating an in-contact skill from a human demonstrationand automatically enhancing the imitated force profile usinga policy search method. The framework encodes both the thedemonstrated trajectory and the normal contact force usingDynamic Movement Primitives (DMPs). In experiments, we utilizePolicy Improvement with Path Integral (PI2) algorithm forupdating the imitated force policy. Our results demonstrate theeffectiveness of this approach in improving the performanceof a wood planing task. After only two update rounds, all theupdated policies have outperformed the imitated policy at asignificance level of P < 0.001.
Original languageEnglish
Title of host publication16th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2016
Number of pages17
ISBN (Electronic)978-1-5090-4718-5
Publication statusPublished - 2 Jan 2017
MoE publication typeA4 Article in a conference publication
EventIEEE-RAS International Conference on Humanoid Robots - Cancun, Mexico
Duration: 15 Nov 201617 Nov 2016
Conference number: 16

Publication series

NameIEEE-RAS International Conference on Humanoid Robots
PublisherInstitute of Electrical and Electronics Engineers, Inc.
ISSN (Print)2164-0572
ISSN (Electronic)2164-0580


ConferenceIEEE-RAS International Conference on Humanoid Robots
Abbreviated titleHumanoids 2016


  • learning from demonstration
  • programming by demonstration
  • in-contact task
  • in-contact
  • skill optimization
  • reinforcement learning

Fingerprint Dive into the research topics of 'Reinforcement learning for improving imitated in-contact skills'. Together they form a unique fingerprint.

Cite this