Co-imagination of Behaviour and Morphology of Agents

Maria Sliacka, Michael Mistry, Roberto Calandra, Ville Kyrki, Kevin Sebastian Luck*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

Abstract

The field of robot learning has made great advances in developing behaviour learning methodologies capable of learning policies for tasks ranging from manipulation to locomotion. However, the problem of combined learning of behaviour and robot structure, here called co-adaptation, is less studied. Most of the current co-adapting robot learning approaches rely on model-free algorithms or assume to have access to an a-priori known dynamics model, which requires considerable human engineering. In this work, we investigate the potential of combining model-free and model-based reinforcement learning algorithms for their application on co-adaptation problems with unknown dynamics functions. Classical model-based reinforcement learning is concerned with learning the forward dynamics of a specific agent or robot in its environment. However, in the case of jointly learning the behaviour and morphology of agents, each individual agent-design implies its own specific dynamics function. Here, the challenge is to learn a dynamics model capable of generalising between the different individual dynamics functions or designs. In other words, the learned dynamics model approximates a multi-dynamics function with the goal to generalise between different agent designs. We present a reinforcement learning algorithm that uses a learned multi-dynamics model for co-adapting robot’s behaviour and morphology using imagined rollouts. We show that using a multi-dynamics model for imagining transitions can lead to better performance for model-free co-adaptation, but open challenges remain.

Original languageEnglish
Title of host publicationMachine Learning, Optimization, and Data Science - 9th International Conference, LOD 2023
EditorsGiuseppe Nicosia, Varun Ojha, Emanuele La Malfa, Gabriele La Malfa, Panos M. Pardalos, Renato Umeton
PublisherSpringer
Pages318-332
Number of pages15
ISBN (Print)978-3-031-53968-8
DOIs
Publication statusPublished - 2024
MoE publication typeA4 Conference publication
EventInternational Conference on Machine Learning, Optimization, and Data Science - Grasmere, United Kingdom
Duration: 22 Sept 202326 Sept 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14505 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Conference on Machine Learning, Optimization, and Data Science
Abbreviated titleLOD
Country/TerritoryUnited Kingdom
CityGrasmere
Period22/09/202326/09/2023

Keywords

  • Co-Adaptation
  • Co-Design
  • Evolutionary Robotics
  • Reinforcement Learning

Fingerprint

Dive into the research topics of 'Co-imagination of Behaviour and Morphology of Agents'. Together they form a unique fingerprint.

Cite this