Sparse latent space policy search

Kevin Sebastian Luck, Joni Pajarinen, Erik Berger, Ville Kyrki, Heni Ben Amor

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

7 Citations (Scopus)

Abstract

Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sample efficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method uses Variational Inference to estimate policy parameters, while at the same time uncovering a low-dimensional latent space of controls. Prior knowledge about the task and the structure of the learning agent can be provided by specifying groups of potentially correlated parameters. This information is then used to impose sparsity constraints on the mapping between the high-dimensional space of controls and a lowerdimensional latent space. In experiments with a simulated bi-manual manipulator, the new approach effectively identifies synergies between joints, performs efficient low-dimensional policy search, and outperforms state-of-the-art policy search methods.

Original languageEnglish
Title of host publicationProceedings of the 30th AAAI Conference on Artificial Intelligence, AAAI 2016
PublisherAAAI PRESS
Pages1911-1918
Number of pages8
ISBN (Electronic)9781577357605
Publication statusPublished - 2016
MoE publication typeA4 Article in a conference publication
EventAAAI Conference on Artificial Intelligence - Phoenix, United States
Duration: 12 Feb 201617 Feb 2016
Conference number: 30

Publication series

NameProceedings of the AAAI Conference on Artificial Intelligence
PublisherAAAI PRESS
ISSN (Print)2159-5399
ISSN (Electronic)2374-3468

Conference

ConferenceAAAI Conference on Artificial Intelligence
Abbreviated titleAAAI
CountryUnited States
CityPhoenix
Period12/02/201617/02/2016

Fingerprint Dive into the research topics of 'Sparse latent space policy search'. Together they form a unique fingerprint.

Cite this