Crowdsourced 3D semantic mapping and change detection in urban driving environments

The advancement of autonomous driving hinges on the availability of accurate and up-to-date semantic maps, which provide a detailed representation of the environment for safe and efficient route planning, obstacle avoidance, and decision-making in urban environment. In order to manage the dynamic nature of these environments, which are frequently modified by changes such as updated traffic signs, traditional methods of generating and updating semantic maps, involve specialized vehicles equipped with high-precision sensors, which are both time-consuming and costly. To mitigate these challenges, this dissertation investigates the potential of leveraging crowdsourced image data from consumer-grade cameras, such as smartphones and dashboard cameras, as well as traffic flow data, which can be obtained from existing mapping services like HERE Maps. In this dissertation, we first introduce a novel pipeline designed to automatically detect changes (e.g., types and locations of traffic signs) in complex, large-scale urban driving environments, utilizing point clouds generated from visual data using Structure from Motion (SfM). The pipeline comprises several components, each addressing specific challenges such as online change detection, and accurate change localization in 3D space. Second, we provide key guidelines for constructing large-scale 3D maps using visual crowdsourced data to ensure both accuracy and reliability. We examine the challenges posed by the inherent monocular nature and data inconsistency of visual crowdsourcing data, and conduct a comprehensive comparison of various SfM techniques on such data in complex, large-scale urban environments. Finally, we propose a deep-learning method for coarse-grained change detection using traffic flow data to reduce the costly and extensive search space required to detect changes in large-scale environments. In practice, this coarse-grained change detection can be used to initially identify areas of change, which can then be refined using the pipeline from the first step for precise localization and semantic map updates. The results of this work provide the basis for the future deployment of cost-effective automatic change detection for accurate and up-to-date semantic maps for autonomous driving.