Imitation-guided Multimodal Policy Generation from Behaviourally Diverse Demonstrations

Shibei Zhu*, Rituraj Kaushik, Samuel Kaski, Ville Kyrki

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

48 Downloads (Pure)

Abstract

Learning policies from multiple demonstrators is often difficult because different individuals perform the same task differently due to hidden factors such as preferences. In the context of policy learning, this leads to multimodal policies. Existing policy learning methods often converge to a single solution mode, failing to capture the diversity in the solution space. In this paper, we introduce an imitation-guided reinforcement learning framework to solve the multimodal policy learning problem from a limited number of state-only demonstrations. Then, we propose LfBD (Learning from Behaviourally diverse Demonstration), an algorithm that builds a parameterised solution space to capture the variability in the behaviour space defined by demonstrations. To this end, we define a projection function based on the state density distributions from demonstrations to define such space. Our goal is not only to learn how to solve the task as the human demonstrator but also to extrapolate beyond the provided demonstrations. In addition, we show that with our method, we can perform a post-hoc policy search in the built solution space to recover policies that satisfy specific constraints or to find a policy that matches a given (state-only) behaviour.
Original languageEnglish
Title of host publication2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
PublisherIEEE
Pages1675-1682
Number of pages8
ISBN (Electronic)978-1-6654-9190-7
DOIs
Publication statusPublished - 2023
MoE publication typeA4 Conference publication
EventIEEE/RSJ International Conference on Intelligent Robots and Systems
- Detroit, United States
Duration: 1 Oct 20235 Oct 2023

Publication series

NameProceedings of the IEEE/RSJ international conference on intelligent robots and systems
PublisherIEEE
ISSN (Electronic)2153-0866

Conference

ConferenceIEEE/RSJ International Conference on Intelligent Robots and Systems
Abbreviated titleIROS
Country/TerritoryUnited States
CityDetroit
Period01/10/202305/10/2023

Keywords

  • Reinforcement Learning
  • Learning from Demonstration
  • User modelling

Fingerprint

Dive into the research topics of 'Imitation-guided Multimodal Policy Generation from Behaviourally Diverse Demonstrations'. Together they form a unique fingerprint.

Cite this