Projects per year
Abstract
Online planning is crucial for high performance in many complex sequential decision-making tasks. Monte Carlo Tree Search (MCTS) employs a principled mechanism for trading off exploration for exploitation for efficient online planning, and it outperforms comparison methods in many discrete decision-making domains such as Go, Chess, and Shogi. Subsequently, extensions of MCTS to continuous domains have been developed. However, the inherent high branching factor and the resulting explosion of the search tree size are limiting the existing methods. To address this problem, we propose Continuous Monte Carlo Graph Search (CMCGS), an extension of MCTS to online planning in environments with continuous state and action spaces. CMCGS takes advantage of the insight that, during planning, sharing the same action policy between several states can yield high performance. To implement this idea, at each time step, CMCGS clusters similar states into a limited number of stochastic action bandit nodes, which produce a layered directed graph instead of an MCTS search tree. Experimental evaluation shows that CMCGS outperforms comparable planning methods in several complex continuous DeepMind Control Suite benchmarks and 2D navigation and exploration tasks with limited sample budgets. Furthermore, CMCGS can be scaled up through parallelization, and it outperforms the Cross-Entropy Method (CEM) in continuous control with learned dynamics models.
Original language | English |
---|---|
Title of host publication | AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems |
Publisher | ACM |
Pages | 1047-1056 |
Number of pages | 10 |
ISBN (Print) | 979-8-4007-0486-4 |
DOIs | |
Publication status | Published - 2024 |
MoE publication type | A4 Conference publication |
Event | International Conference on Autonomous Agents and Multiagent Systems - Auckland, New Zealand Duration: 6 May 2024 → 10 May 2024 |
Conference
Conference | International Conference on Autonomous Agents and Multiagent Systems |
---|---|
Abbreviated title | AAMAS |
Country/Territory | New Zealand |
City | Auckland |
Period | 06/05/2024 → 10/05/2024 |
Keywords
- Continuous Control
- MCTS
- Model-Based Reinforcement Learning
- Online Planning
- Planning
- Reinforcement Learning
Fingerprint
Dive into the research topics of 'Continuous Monte Carlo Graph Search'. Together they form a unique fingerprint.-
Safe: Turvallinen vahvistusoppiminen epästationaarisissa ympäristöissä nopealla sopeutumisella ja häiriöennustuksella
Pajarinen, J. (Principal investigator), Kostin, N. (Project Member) & Zhao, Y. (Project Member)
01/01/2022 → 31/12/2024
Project: Academy of Finland: Other research funding
-
-: Finnish Center for Artificial Intelligence
Kaski, S. (Principal investigator)
01/01/2019 → 31/12/2022
Project: Academy of Finland: Other research funding