Behaviour-Conditioned Policies for Cooperative Reinforcement Learning Tasks

Antti Keurulainen*, Isak Westerlund, Ariel Kwiatkowski, Samuel Kaski, Alexander Ilin

*Tämän työn vastaava kirjoittaja

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

Abstrakti

The cooperation among AI systems, and between AI systems and humans is becoming increasingly important. In various real-world tasks, an agent needs to cooperate with unknown partner agent types. This requires the agent to assess the behaviour of the partner agent during a cooperative task and to adjust its own policy to support the cooperation. Deep reinforcement learning models can be trained to deliver the required functionality but are known to suffer from sample inefficiency and slow learning. However, adapting to a partner agent behaviour during the ongoing task requires ability to assess the partner agent type quickly. We suggest a method, where we synthetically produce populations of agents with different behavioural patterns together with ground truth data of their behaviour, and use this data for training a meta-learner. We additionally suggest an agent architecture, which can efficiently use the generated data and gain the meta-learning capability. When an agent is equipped with such a meta-learner, it is capable of quickly adapting to cooperation with unknown partner agent types in new situations. This method can be used to automatically form a task distribution for meta-training from emerging behaviours that arise, for example, through self-play.

AlkuperäiskieliEnglanti
OtsikkoArtificial Neural Networks and Machine Learning – ICANN 2021 - 30th International Conference on Artificial Neural Networks, Proceedings
ToimittajatIgor Farkaš, Paolo Masulli, Sebastian Otte, Stefan Wermter
Sivut493-504
Sivumäärä12
DOI - pysyväislinkit
TilaJulkaistu - 2021
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaInternational Conference on Artificial Neural Networks - Virtual, Online
Kesto: 14 syyskuuta 202117 syyskuuta 2021
Konferenssinumero: 30

Julkaisusarja

NimiLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Vuosikerta12894 LNCS
ISSN (painettu)0302-9743
ISSN (elektroninen)1611-3349

Conference

ConferenceInternational Conference on Artificial Neural Networks
LyhennettäICANN
KaupunkiVirtual, Online
Ajanjakso14/09/202117/09/2021

Sormenjälki

Sukella tutkimusaiheisiin 'Behaviour-Conditioned Policies for Cooperative Reinforcement Learning Tasks'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä