From Video Game to Real Robot: The Transfer between Action Spaces

Janne Karttunen, Anssi Kanervisto, Ville Kyrki, Ville Hautamaki

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Deep reinforcement learning has proven to be successful for learning tasks in simulated environments, but applying same techniques for robots in real-world domain is more challenging, as they require hours of training. To address this, transfer learning can be used to train the policy first in a simulated environment and then transfer it to physical agent. As the simulation never matches reality perfectly, the physics, visuals and action spaces by necessity differ between these environments to some degree. In this work, we study how general video games can be directly used instead of fine-tuned simulations for the sim-to-real transfer. Especially, we study how the agent can learn the new action space autonomously, when the game actions do not match the robot actions. Our results show that the different action space can be learned by re-training only part of neural network and we obtain above 90% mean success rate in simulation and robot experiments.

Original languageEnglish
Title of host publicationProceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020
PublisherIEEE
Pages3567-3571
Number of pages5
ISBN (Electronic)9781509066315
DOIs
Publication statusPublished - May 2020
MoE publication typeA4 Article in a conference publication
EventIEEE International Conference on Acoustics, Speech, and Signal Processing - Virtual conference, Barcelona, Spain
Duration: 4 May 20208 May 2020
Conference number: 45

Publication series

NameProceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing
PublisherIEEE
ISSN (Print)1520-6149
ISSN (Electronic)2379-190X

Conference

ConferenceIEEE International Conference on Acoustics, Speech, and Signal Processing
Abbreviated titleICASSP
CountrySpain
CityBarcelona
Period04/05/202008/05/2020
OtherVirtual conference

Keywords

  • Action space transfer
  • Deep reinforcement learning
  • Reality gap
  • Sim-to-real
  • Transfer learning

Fingerprint

Dive into the research topics of 'From Video Game to Real Robot: The Transfer between Action Spaces'. Together they form a unique fingerprint.

Cite this