Understanding and predicting human behavior in interactions with autonomous systems in urban environments: a systematic review, challenges, and opportunities

Urban transportation is undergoing a transformative shift with the advent of autonomous systems. Consequently, understanding and predicting the behaviors of vulnerable human road users (i.e., pedestrians and cyclists), including their intentions, decisions, and movements when interacting with autonomous systems becomes crucial. Reviewing the literature spanning the years 2014 to 2023, we identified 119 articles that empirically investigate the microscopic human-autonomous systems interaction (HAI). Through a systematic analysis, our paper offers a holistic overview of progress and challenges within this field, from the initial data collection and experiment design to its modeling methodologies. Based on our findings, we present our vision of bridging experimental HAIresearch and the works applying computer vision techniques for real-world pedestrian behavior analysis, offering valuable insights for future research directions.