Integrating Multi-Purpose Natural Language Understanding, Robot's Memory, and Symbolic Planning for Task Execution in Humanoid Robots

Mirko Wächter, Ekaterina Ovchinnikova, Valerij Wittenbeck, Peter Kaiser, Sandor Szedmak, Wail Mustafa, Dirk Kraft, Norbert Krüger , Justus Piater, Tamim Asfour

Research output: Contribution to journalArticleScientificpeer-review

Abstract

We propose an approach for instructing a robot using natural language to solve complex tasks in a dynamic environment. In this study, we elaborate on a framework that allows a humanoid robot to understand natural language, derive symbolic representations of its sensorimotor experience, generate complex plans according to the current world state, and monitor plan execution. The presented development supports replacing missing objects and suggesting possible object locations. It is a realization of the concept of structural bootstrapping developed in the context of the European project Xperience. The framework is implemented within the robot development environment ArmarX. We evaluate the framework on the humanoid robot ARMAR-III in the context of two experiments: a demonstration of the real execution of a complex task in the kitchen environment on ARMAR-III and an experiment with untrained users in a simulation environment.
Original languageEnglish
Pages (from-to)148-165
JournalRobotics and Autonomous Systems
Volume99
DOIs
Publication statusPublished - 2018
MoE publication typeA1 Journal article-refereed

Keywords

  • Structural bootstrapping
  • Natural language understanding
  • Planning
  • Task execution
  • Object replacement
  • Humanoid robotics

Fingerprint

Dive into the research topics of 'Integrating Multi-Purpose Natural Language Understanding, Robot's Memory, and Symbolic Planning for Task Execution in Humanoid Robots'. Together they form a unique fingerprint.

Cite this