Model-based robotic hand tracking and gripper state determination

Arthur Cruzde Araujo, Antonio Marcus Nogueira Lima, Jaakko Mikael Mattila, Rajkumar Muthusamy

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


This paper proposes a 3D model-based vision system for detecting a manipulator's gripper on a scene, tracking it as it moves and continuously determining its state (opened or closed), information that is used to signal that an object can be grasped. Build upon the Robot Operating System (ROS), it mainly comprises a registration pipeline, a tracking algorithm and a state classifier based on a proposed measure of similarity between point clouds. The test platform includes a robotic arm and a depth camera. Satisfactory experimental results were obtained: tracking had a good performance for well-behaved trajectories and the detection of not only the state of the gripper, showing robustness to self-occlusion, but also the presence of an object, were successful. The state detector produced relevant output on grasping of unknown objects to be used in future works, which might contemplate the insertion of this system in a multi-robot collaborative scenario. The test results presented in this paper demonstrate the correctness of the proposed methodology and the feasibility of the proposed system design.

Original languageEnglish
Title of host publicationProceedings of the 2017 14th Latin American Robotics Symposium LARS and 2017 5th Brazilian Symposium on Robotics SBR
EditorsE. Todt, F. Tonidandel
Number of pages6
ISBN (Electronic)9781538609569
Publication statusPublished - 15 Dec 2017
MoE publication typeA4 Article in a conference publication
EventLatin American Robotics Symposium and Brazilian Robotics Symposium - Curitiba, Brazil
Duration: 8 Nov 201711 Nov 2017


ConferenceLatin American Robotics Symposium and Brazilian Robotics Symposium
Abbreviated titleLARS-SBR


  • CAD-based vision
  • model-based
  • registration
  • robotic manipulators
  • ROS
  • tracking

Fingerprint Dive into the research topics of 'Model-based robotic hand tracking and gripper state determination'. Together they form a unique fingerprint.

Cite this