Comparing Reinforcement Learning Methods for Computational Curiosity through Behavioural Analysis

Curiosity, a desire to know or learn more, appears to motivate many of the decisions made by biological systems. Numerous researchers, curious about a computational equivalent, have experimented with the idea of using computational reinforcement learning to produce curious behaviour in learning agents. Their efforts have resulted in a rich foundation for future work on curiosity, but the relationships between existing methods remain poorly understood. We suggest that one way to solidify this foundation is through a comparison of the behaviours resulting from different curiosity methods. In a domain with clear properties, the same agent, when motivated by different computational curiosity methods, will follow different behavioural trajectories. Tracking the underlying changes in such a curious agent’s computations allows us to clarify why its behaviours differ and better understand how agents motivated by the tested methods might behave overall. Given the clear importance of understanding curiosity in understanding the decision-making behaviour of both biological and artificially intelligent systems, we emphasize the relevance of a systematic study of computationally curious behaviours and suggest that it is natural to begin with reinforcement learning methods.