Learning High-Risk High-Precision Motion Control

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

49 Downloads (Pure)

Abstract

Deep reinforcement learning (DRL) algorithms for movement control are typically evaluated and benchmarked on sequential decision tasks where imprecise actions may be corrected with later actions, thus allowing high returns with noisy actions. In contrast, we focus on an under-researched class of high-risk, high-precision motion control problems where actions carry irreversible outcomes, driving sharp peaks and ridges to plague the state-action reward landscape. Using computational pool as a representative example of such problems, we propose and evaluate State-Conditioned Shooting (SCOOT), a novel DRL algorithm that builds on advantage-weighted regression (AWR) with three key modifications: 1) Performing policy optimization only using elite samples, allowing the policy to better latch on to the rare high-reward action samples; 2) Utilizing a mixture-of-experts (MoE) policy, to allow switching between reward landscape modes depending on the state; 3) Adding a distance regularization term and a learning curriculum to encourage exploring diverse strategies before adapting to the most advantageous samples. We showcase our features’ performance in learning physically-based billiard shots demonstrating high action precision and discovering multiple shot strategies for a given ball configuration.
Original languageEnglish
Title of host publicationMIG '22: Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games
EditorsStephen N. Spencer
PublisherACM
Pages1-10
Number of pages10
ISBN (Electronic)978-1-4503-9888-6
DOIs
Publication statusPublished - 3 Nov 2022
MoE publication typeA4 Conference publication
EventACM SIGGRAPH Conference on Motion, Interaction and Games - Guanajuato, Mexico
Duration: 3 Nov 20225 Nov 2022
Conference number: 15

Conference

ConferenceACM SIGGRAPH Conference on Motion, Interaction and Games
Abbreviated titleMIG
Country/TerritoryMexico
City Guanajuato
Period03/11/202205/11/2022

Fingerprint

Dive into the research topics of 'Learning High-Risk High-Precision Motion Control'. Together they form a unique fingerprint.

Cite this