TY - GEN
T1 - Co-imagination of Behaviour and Morphology of Agents
AU - Sliacka, Maria
AU - Mistry, Michael
AU - Calandra, Roberto
AU - Kyrki, Ville
AU - Luck, Kevin Sebastian
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
PY - 2024
Y1 - 2024
N2 - The field of robot learning has made great advances in developing behaviour learning methodologies capable of learning policies for tasks ranging from manipulation to locomotion. However, the problem of combined learning of behaviour and robot structure, here called co-adaptation, is less studied. Most of the current co-adapting robot learning approaches rely on model-free algorithms or assume to have access to an a-priori known dynamics model, which requires considerable human engineering. In this work, we investigate the potential of combining model-free and model-based reinforcement learning algorithms for their application on co-adaptation problems with unknown dynamics functions. Classical model-based reinforcement learning is concerned with learning the forward dynamics of a specific agent or robot in its environment. However, in the case of jointly learning the behaviour and morphology of agents, each individual agent-design implies its own specific dynamics function. Here, the challenge is to learn a dynamics model capable of generalising between the different individual dynamics functions or designs. In other words, the learned dynamics model approximates a multi-dynamics function with the goal to generalise between different agent designs. We present a reinforcement learning algorithm that uses a learned multi-dynamics model for co-adapting robot’s behaviour and morphology using imagined rollouts. We show that using a multi-dynamics model for imagining transitions can lead to better performance for model-free co-adaptation, but open challenges remain.
AB - The field of robot learning has made great advances in developing behaviour learning methodologies capable of learning policies for tasks ranging from manipulation to locomotion. However, the problem of combined learning of behaviour and robot structure, here called co-adaptation, is less studied. Most of the current co-adapting robot learning approaches rely on model-free algorithms or assume to have access to an a-priori known dynamics model, which requires considerable human engineering. In this work, we investigate the potential of combining model-free and model-based reinforcement learning algorithms for their application on co-adaptation problems with unknown dynamics functions. Classical model-based reinforcement learning is concerned with learning the forward dynamics of a specific agent or robot in its environment. However, in the case of jointly learning the behaviour and morphology of agents, each individual agent-design implies its own specific dynamics function. Here, the challenge is to learn a dynamics model capable of generalising between the different individual dynamics functions or designs. In other words, the learned dynamics model approximates a multi-dynamics function with the goal to generalise between different agent designs. We present a reinforcement learning algorithm that uses a learned multi-dynamics model for co-adapting robot’s behaviour and morphology using imagined rollouts. We show that using a multi-dynamics model for imagining transitions can lead to better performance for model-free co-adaptation, but open challenges remain.
KW - Co-Adaptation
KW - Co-Design
KW - Evolutionary Robotics
KW - Reinforcement Learning
UR - http://www.scopus.com/inward/record.url?scp=85187670267&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-53969-5_24
DO - 10.1007/978-3-031-53969-5_24
M3 - Conference article in proceedings
AN - SCOPUS:85187670267
SN - 978-3-031-53968-8
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 318
EP - 332
BT - Machine Learning, Optimization, and Data Science - 9th International Conference, LOD 2023
A2 - Nicosia, Giuseppe
A2 - Ojha, Varun
A2 - La Malfa, Emanuele
A2 - La Malfa, Gabriele
A2 - Pardalos, Panos M.
A2 - Umeton, Renato
PB - Springer
T2 - International Conference on Machine Learning, Optimization, and Data Science
Y2 - 22 September 2023 through 26 September 2023
ER -