Benchmarking of airborne laser scanning based feature extraction methods and mobile laser scanning system performance based on high-quality test fields

Comparing different feature extraction methods based on remote sensing or remote sensing systems is difficult as there are but few common data sets or test fields with reference data of high standard available for analysis. State-of-the-art methods and systems are often in still evolving stage and can be run only by the developers themselves. Establishing a high-quality test field is laborious, but once such a test field has been established, it becomes easier to set up the systems to collect data from the field than to collect reference data from new areas. Comparing either different systems or the same system with different parameters is easier when the number of variables is kept to a minimum; the remotely sensed areas are kept constant and any changes in them can be controlled more easily. The benchmarking results provide valuable information to both developers and users of remote sensing data products. The benchmarked feature extraction methods studied included extraction of buildings and individual trees using data from common test fields. The performance of the mobile laser scanning systems was benchmarked using data collected from an established urban test field. In all cases, it was concluded that the primary factor affecting the results was the method or the system, and this enabled a high degree of comparability for the results of the given extraction or mapping tasks.