Open Ad Hoc Teamwork with Cooperative Game Theory

Jianhong Wang*, Yang Li, Yuan Zhang, Wei Pan, Samuel Kaski

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

Abstract

Ad hoc teamwork poses a challenging problem, requiring the design of an agent to collaborate with teammates without prior coordination or joint training. Open ad hoc teamwork (OAHT) further complicates this challenge by considering environments with a changing number of teammates, referred to as open teams. One promising solution in practice to this problem is leveraging the generalizability of graph neural networks to handle an unrestricted number of agents with various agent-types, named graph-based policy learning (GPL). However, its joint Q-value representation over a coordination graph lacks convincing explanations. In this paper, we establish a new theory to understand the representation of the joint Q-value for OAHT and its learning paradigm, through the lens of cooperative game theory. Building on our theory, we propose a novel algorithm named CIAO, based on GPL’s framework, with additional provable implementation tricks that can facilitate learning. The demos of experimental results are available on https://sites.google.com/view/ciao2024, and the code of experiments is published on https://github.com/hsvgbkhgbv/CIAO.
Original languageEnglish
Title of host publicationProceedings of the 41st International Conference on Machine Learning
EditorsRuslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, Felix Berkenkamp
PublisherJMLR
Pages50902-50930
Number of pages29
Volume235
Publication statusPublished - 2024
MoE publication typeA4 Conference publication
EventInternational Conference on Machine Learning - Vienna, Austria
Duration: 21 Jul 202427 Jul 2024
Conference number: 41

Publication series

NameProceedings of Machine Learning Research
PublisherJMLR
Volume235
ISSN (Electronic)2640-3498

Conference

ConferenceInternational Conference on Machine Learning
Abbreviated titleICML
Country/TerritoryAustria
CityVienna
Period21/07/202427/07/2024

Fingerprint

Dive into the research topics of 'Open Ad Hoc Teamwork with Cooperative Game Theory'. Together they form a unique fingerprint.

Cite this