Inverse reinforcement learning from summary data

Tutkimustuotos: Lehtiartikkelivertaisarvioitu

Tutkijat

Organisaatiot

Kuvaus

Inverse reinforcement learning (IRL) aims to explain observed strategic behavior by fitting reinforcement learning models to behavioral data. However, traditional IRL methods are only applicable when the observations are in the form of state-action paths. This assumption may not hold in many real-world modeling settings, where only partial or summarized observations are available. In general, we may assume that there is a summarizing function σ , which acts as a filter between us and the true state-action paths that constitute the demonstration. Some initial approaches to extending IRL to such situations have been presented, but with very specific assumptions about the structure of σ , such as that only certain state observations are missing. This paper instead focuses on the most general case of the problem, where no assumptions are made about the summarizing function, except that it can be evaluated. We demonstrate that inference is still possible. The paper presents exact and approximate inference algorithms that allow full posterior inference, which is particularly important for assessing parameter uncertainty in this challenging inference situation. Empirical scalability is demonstrated to reasonably sized problems, and practical applicability is demonstrated by estimating the posterior for a cognitive science RL model based on an observed user’s task completion time only.

Yksityiskohdat

AlkuperäiskieliEnglanti
Sivut1517–1535
JulkaisuMachine Learning
Vuosikerta107
Numero8-10
TilaJulkaistu - syyskuuta 2018
OKM-julkaisutyyppiA1 Julkaistu artikkeli, soviteltu
TapahtumaEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - Riva del Garda, Italia
Kesto: 19 syyskuuta 201623 syyskuuta 2016

ID: 26236039