Robot-Robot Gesturing for Anchoring Representations

Research output: Contribution to journalArticle


Research units

  • Blue Ocean Robotics
  • Université Paris-Saclay
  • University College London


In a multirobot system, using shared symbols for objects in the environment is a prerequisite for collaboration. Sharing symbols requires that each agent has anchored a symbol with an internal, sensor level representation, as well as that these symbols match between the agents. The problem can be solved easily when the internal representations can be communicated between the agents. However, with heterogeneous embodiments the available sensors are likely to differ, making it impossible to share the internal representations directly. We propose the use of pointing gestures to align symbols between a heterogeneous group of robots. We describe a planning framework that minimizes the required effort for anchoring representations across robots. The framework allows planning for both the gesturing and observing agents in a decentralized fashion. It considers both implicit sources of failure, such as ambiguous pointing, as well as costs required by actions. Simulation experiments demonstrate that the resulting planning problem has a complex solution structure with multiple local minima. Demonstration with a heterogeneous two-robot system shows the practical viability of this approach.


Original languageEnglish
Article number8502843
Pages (from-to)216-230
Number of pages15
JournalIEEE Transactions on Robotics
Issue number1
Early online dateOct 2018
Publication statusPublished - 1 Feb 2019
MoE publication typeA1 Journal article-refereed

    Research areas

  • Cognitive robotics, Grounding, Multi-robot systems, multi-robot systems, Planning, Robot kinematics, Robot sensing systems, symbol grounding, Tracking

Download statistics

No data available

ID: 29462620