Projekteja vuodessa
Abstrakti
Visual-language models (VLMs) have recently been introduced in robotic mapping by using the latent representations, i.e., embeddings, of the VLMs to represent the natural language semantics in the map. The main benefit is moving beyond a small set of human-created labels toward open-vocabulary scene understanding. While there is anecdotal evidence that maps built this way support downstream tasks, such as navigation, rigorous analysis of the quality of the maps using these embeddings is lacking. We investigate two critical properties of map quality: queryability and consistency. The evaluation of queryability addresses the ability to retrieve information from the embeddings. We investigate two aspects of consistency: intra-map consistency and inter-map consistency. Intra-map consistency captures the ability of the embeddings to represent abstract semantic classes, and inter-map consistency captures the generalization properties of the representation. In this paper, we propose a way to analyze the quality of maps created using VLMs, which forms an open-source benchmark to be used when proposing new open-vocabulary map representations. We demonstrate the benchmark by evaluating the maps created by two state-of-the-art methods, VLMaps and OpenScene, using two encoders, LSeg and OpenSeg, using real-world data from the Matterport3D data set. We find that OpenScene outperforms VLMaps with both encoders, and LSeg outperforms OpenSeg with both methods.
Alkuperäiskieli | Englanti |
---|---|
Otsikko | Proceedings of IEEE-RAS International Conference on Humanoid Robots |
DOI - pysyväislinkit | |
Tila | Valmisteltavana - 2024 |
OKM-julkaisutyyppi | A4 Artikkeli konferenssijulkaisussa |
Tapahtuma | IEEE-RAS International Conference on Humanoid Robots - Nancy, Ranska Kesto: 22 marrask. 2024 → 24 marrask. 2024 https://2024.ieee-humanoids.org/ |
Conference
Conference | IEEE-RAS International Conference on Humanoid Robots |
---|---|
Maa/Alue | Ranska |
Kaupunki | Nancy |
Ajanjakso | 22/11/2024 → 24/11/2024 |
www-osoite |
Sormenjälki
Sukella tutkimusaiheisiin 'Do Visual-Language Maps Capture Latent Semantics?'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.-
Hypermaps: Hypermaps: closing the complexity gap in robotic mapping
Verdoja, F. (Vastuullinen tutkija) & Nguyen, P. (Projektin jäsen)
01/09/2023 → 31/08/2027
Projekti: Academy of Finland: Other research funding
-
SANTTU: Kumppanuusmalli - SANTTU - Aalto
Kyrki, V. (Vastuullinen tutkija), Chaubey, S. (Projektin jäsen), Verdoja, F. (Projektin jäsen), Pekkanen, M. (Projektin jäsen), Nguyen Le, T. (Projektin jäsen), Arndt, K. (Projektin jäsen), Struckmeier, O. (Projektin jäsen), Hannus, E. (Projektin jäsen) & Blanco Mulero, D. (Projektin jäsen)
01/04/2022 → 31/03/2024
Projekti: Business Finland: Strategic centres for science, technology and innovation (SHOK)