Training and Evaluation of Deep Policies Using Reinforcement Learning and Generative Models

Ali Ghadirzadeh, Petra Poklukar*, Karol Arndt, Chelsea Finn, Ville Kyrki, Danica Kragic, Mårten Björkman

*Tämän työn vastaava kirjoittaja

Tutkimustuotos: LehtiartikkeliArticleScientificvertaisarvioitu

10 Lataukset (Pure)

Abstrakti

We present a data-efficient framework for solving sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models. The framework, called GenRL, trains deep policies by introducing an action latent variable such that the feed-forward policy search can be divided into two parts: (i) training a sub-policy that outputs a distribution over the action latent variable given a state of the system, and (ii) unsupervised training of a generative model that outputs a sequence of motor actions conditioned on the latent action variable. GenRL enables safe exploration and alleviates the data-inefficiency problem as it exploits prior knowledge about valid sequences of motor actions. Moreover, we provide a set of measures for evaluation of generative models such that we are able to predict the performance of the RL policy training prior to the actual training on a physical robot. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training on two robotics tasks: shooting a hockey puck and throwing a basketball. Furthermore, we empirically demonstrate that GenRL is the only method which can safely and efficiently solve the robotics tasks compared to two state-of-the-art RL methods.
AlkuperäiskieliEnglanti
Artikkeli174
Sivut1-37
Sivumäärä37
JulkaisuJournal of Machine Learning Research
Vuosikerta23
Numero174
TilaJulkaistu - 4 elok. 2022
OKM-julkaisutyyppiA1 Julkaistu artikkeli, soviteltu

Sormenjälki

Sukella tutkimusaiheisiin 'Training and Evaluation of Deep Policies Using Reinforcement Learning and Generative Models'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä