Image-based Renovation Progress Inspection with Deep Siamese Networks

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

21 Downloads (Pure)

Abstract

Various specialized machine vision systems have been proposed for item inspection, robot supervision and quality control in manufacturing and construction industries. However, construction industries are still lacking solutions for automating the progress inspection in building renovation projects. As smartphones becoming increasingly pervasive among construction workers and use of smartphone photos for documentation getting more popular, in this work, we propose machine learning methods for automatic progress recognition in renovation projects from smartphone images. Renovation progress inspection is formulated as an ordinal classification problem, with every class representing one stage of the renovation process. The baseline solutions inspired by the popular deep learning architectures like VGG19, ResNet, Xception, DenseNet and MobileNet do not benefit from the temporal property of the data. Consecutive stages share a substantial amount of visual features, which makes it difficult to differentiate between them. To cope with that, we design special networks - Ordering Nets which utilize the consecutive property of the classes to predict the order of a photo pair. For the special use case, we train the 1-Step- Net to recognize progress from two subsequent photos. The extracted order information increases classification accuracy and provides more precise temporal prediction. We report our results on a new Reno-2018 dataset - a two-part collection of photos that covers bathroom and kitchen renovation steps. The applicability of our approach is demonstrated on a simulated progress estimation task. Our method significantly improves the temporal accuracy of stage predictions compared to the base deep neural network models.

Original languageEnglish
Title of host publicationProceedings of the 2020 12th International Conference on Machine Learning and Computing, ICMLC 2020
PublisherACM
Pages96-104
Number of pages9
ISBN (Electronic)9781450376426
DOIs
Publication statusPublished - 15 Feb 2020
MoE publication typeA4 Article in a conference publication
EventInternational Conference on Machine Learning and Computing - Shenzhen, China
Duration: 15 Feb 202017 Feb 2020
Conference number: 12

Conference

ConferenceInternational Conference on Machine Learning and Computing
Abbreviated titleICMLC
CountryChina
CityShenzhen
Period15/02/202017/02/2020

Keywords

  • deep learning
  • Inspection
  • ordinal classification
  • progress estimation

Fingerprint Dive into the research topics of 'Image-based Renovation Progress Inspection with Deep Siamese Networks'. Together they form a unique fingerprint.

  • Activities

    • 1 Participant of a conference, workshop, session or tutorial

    International Conference on Machine Learning and Computing

    Petr Byvshev (Member)

    19 Jun 202021 Jun 2020

    Activity: Participating in or organising an event typesParticipant of a conference, workshop, session or tutorial

    File

    Cite this

    Byvshev, P., Pham, T., & Xiao, Y. (2020). Image-based Renovation Progress Inspection with Deep Siamese Networks. In Proceedings of the 2020 12th International Conference on Machine Learning and Computing, ICMLC 2020 (pp. 96-104). ACM. https://doi.org/10.1145/3383972.3384036