TY - GEN
T1 - A Pilot Study Comparing ChatGPT and Google Search in Supporting Visualization Insight Discovery
AU - He, Chen
AU - Welsch, Robin
AU - Jacucci, Giulio
N1 - Publisher Copyright: © 2024 Copyright for this paper by its authors.; Workshops at the International Conference on Intelligent User Interfaces, IUI-WS 2024 ; Conference date: 18-03-2024 Through 18-03-2024
PY - 2024
Y1 - 2024
N2 - The popularity of large language models (LLMs) provides new possibilities for deriving visualization insights, integrating human and machine intelligence. However, we have yet to understand how a contextualized LLM compares with the traditional search in supporting visualization insight discovery. To this end, we conducted a between-subjects study with 25 participants to compare user insight generation with chat/search on a CO2 Explorer. The Chat condition has ChatGPT contextualized with the data, user tasks, and interactions as programmed system prompts. Results show both systems have their merits and demerits: ChatGPT affords users to ask more diverse questions but can produce wrong answers; Search provides information sources, making the answer more reliable, but users can fail to find the answer. This study prompts us to synthesize them in a future study for reliable and efficient information retrieval.
AB - The popularity of large language models (LLMs) provides new possibilities for deriving visualization insights, integrating human and machine intelligence. However, we have yet to understand how a contextualized LLM compares with the traditional search in supporting visualization insight discovery. To this end, we conducted a between-subjects study with 25 participants to compare user insight generation with chat/search on a CO2 Explorer. The Chat condition has ChatGPT contextualized with the data, user tasks, and interactions as programmed system prompts. Results show both systems have their merits and demerits: ChatGPT affords users to ask more diverse questions but can produce wrong answers; Search provides information sources, making the answer more reliable, but users can fail to find the answer. This study prompts us to synthesize them in a future study for reliable and efficient information retrieval.
KW - Information Visualization
KW - Empirical Study
KW - Google Search
KW - Large Language Models
UR - https://ceur-ws.org/Vol-3660/
M3 - Conference article in proceedings
T3 - CEUR Workshop Proceedings
BT - Joint Proceedings of the ACM IUI Workshops 2024, March 18-21, 2024, Greenville, South Carolina, USA
A2 - Soto, Axel
A2 - Zangerle, Eva
PB - CEUR
CY - Germany
T2 - Workshops at the International Conference on Intelligent User Interfaces
Y2 - 18 March 2024 through 18 March 2024
ER -