2025-4-4- Saturday
HomeIntroductionEditorial boardContact usChinese
Home > Archive>Volume 52, Issue 2, 2024 >213-222. DOI:10.11908/j.issn.0253-374x.22370
PDF HTML XML Export Cite reminder
Deep Learning-Based Computer Vision for Health Monitoring in Civil Engineering
CSTR:
[cstr]
Author:
Affiliation:

1.College of Civil Engineering， Tongji University， Shanghai 200092， China;2.China MCC22 Group Co.， Ltd.， Tangshan 064000， China

Clc Number:

TU317;TP391.4;TU714

  • Article
    • | |
  • Metrics
    • |
  • Reference [55]
    • |
  • Related [20]
    • | | |
  • Comments
    Abstract:

    Health monitoring in the field of civil engineering is of great significance to ensure the long-term and stable service of infrastructure. Compared with traditional monitoring methods， the computer vision technology based on deep learning has the advantages of high efficiency and accuracy. This paper provides a systematic review on the application of the deep learning-based computer vision technology in the field of civil engineering life cycle health monitoring. First， a scientific econometric analysis of the literature in this field is conducted with the help of literature visualization software. Then， the development process of computer vision technology is briefly described， and the methods of data acquisition， data processing， and data annotation in the process of constructing deep learning data sets are summarized. Afterwards， the development and practical engineering application value of the computer vision technology based on deep learning in safety management of construction site， local damage detection of in-service structures and overall damage assessment of structures after disaster are reviewed. Finally， the future application directions are prospected.

    Reference
    [1] CHEN C. CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature [J]. Journal of the American Society for Information Science and Technology, 2006, 57(3): 359.
    [2] FANG C, PING Y, GAO Y, et al. Machine learning-aided multi-objective optimization of structures with hybrid braces-Framework and case study [J]. Engineering Structures, 2022, 269: 114808.
    [3] LECUN Y, BOSER B, DENKER J S, et al. Backpropagation applied to handwritten zip code recognition [J]. Neural Computation, 1989, 1(4): 541.
    [4] YE X W, JIN T, LI Z X, et al. Structural crack detection from benchmark data sets using pruned fully convolutional networks [J]. Journal of Structural Engineering, 2021, 147(11): 04721008.
    [5] FANG Q, LI H, LUO X, et al. Detecting non-hardhat-use by a deep learning method from far-field surveillance videos [J]. Automation in Construction, 2018, 85: 1.
    [6] SHEN J, XIONG X, LI Y, et al. Detecting safety helmet wearing on construction sites with bounding-box regression and deep transfer learning [J]. Computer-Aided Civil and Infrastructure Engineering, 2021, 36(2): 180.
    [7] WANG Z, WU Y, YANG L, et al. Fast personal protective equipment detection for real construction sites using deep learning approaches [J]. Sensors, 2021, 21(10): 3478.
    [8] FANG W, DING L, LUO H, et al. Falls from heights: A computer vision-based approach for safety harness detection [J]. Automation in Construction, 2018, 91: 53.
    [9] XIONG R, TANG P. Machine learning using synthetic images for detecting dust emissions on construction sites [J]. Smart and Sustainable Built Environment, 2021, 10(3): 487.
    [10] DING L, FANG W, LUO H, et al. A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory [J]. Automation in Construction, 2018, 86: 118.
    [11] LUO X, LI H, CAO D, et al. Recognizing diverse construction activities in site images via relevance networks of construction-related objects detected by convolutional neural networks [J]. Journal of Computing in Civil Engineering, 2018, 32(3): 04018012.
    [12] FANG W, DING L, ZHONG B, et al. Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach [J]. Advanced Engineering Informatics, 2018, 37: 139.
    [13] KIM H, KIM H, HONG Y W, et al. Detecting construction equipment using a region-based fully convolutional network and transfer learning [J]. Journal of Computing in Civil Engineering, 2018, 32(2): 04017082.
    [14] LUO H, LIU J, FANG W, et al. Real-time smart video surveillance to manage safety: A case study of a transport mega-project [J]. Advanced Engineering Informatics, 2020, 45: 101100.
    [15] ZENG T, WANG J, CUI B, et al. The equipment detection and localization of large-scale construction jobsite by far-field construction surveillance video based on improving YOLOv3 and grey wolf optimizer improving extreme learning machine [J]. Construction and Building Materials, 2021, 291: 123268.
    [16] CHEN C, ZHU Z, HAMMAD A. Automated excavators activity recognition and productivity analysis from construction site surveillance videos [J]. Automation in Construction, 2020, 110: 103045.
    [17] LIN Z H, CHEN A Y, HSIEH S H. Temporal image analytics for abnormal construction activity identification [J]. Automation in Construction, 2021, 124: 103572.
    [18] LI J, ZHOU G, LI D, et al. Recognizing workers' construction activities on a reinforcement processing area through the position relationship of objects detected by faster R-CNN [J]. Engineering, Construction and Architectural Management, 2023, 30(4): 1657.
    [19] WANG Z, ZHANG Q, YANG B, et al. Vision-based framework for automatic progress monitoring of precast walls by using surveillance videos during the construction phase [J]. Journal of Computing in Civil Engineering, 2021, 35(1): 04020056.
    [20] XIAO B, WANG Y, KANG S C. Deep learning image captioning in construction management: A feasibility study [J]. Journal of Construction Engineering and Management, 2022, 148(7): 04022049.
    [21] ZHU H, GE W, LIU Z. Deep learning-based classification of weld surface defects [J]. Applied Sciences, 2019, 9(16): 3312.
    [22] MERY D, ARTETA C. Automatic defect recognition in x-ray testing using computer vision [C] // 2017 IEEE winter conference on applications of computer vision (WACV). Santa Rosa: IEEE, 2017: 1026-1035.
    [23] SIZYAKIN R, VORONIN V, GAPON N, et al. Automatic detection of welding defects using the convolutional neural network [C] // Automated Visual Inspection and Machine Vision III. Munich: SPIE, 2019, 11061: 93-101.
    [24] YANG L, JIANG H. Weld defect classification in radiographic images using unified deep neural network with multi-level features [J]. Journal of Intelligent Manufacturing, 2021, 32(2): 459.
    [25] YANG D, CUI Y, YU Z, et al. Deep learning based steel pipe weld defect detection [J]. Applied Artificial Intelligence, 2021, 35(15): 1237.
    [26] YUAN C, CHEN W, HAO H, et al. Near real-time bolt-loosening detection using mask and region-based convolutional neural network [J]. Structural Control and Health Monitoring, 2021, 28(7): e2741.
    [27] HUYNH T C, PARK J H, JUNG H J, et al. Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing [J]. Automation in Construction, 2019, 105: 102844.
    [28] QI Y, LI P, XIONG B, et al. A two-step computer vision-based framework for bolt loosening detection and its implementation on a smartphone application [J]. Structural Health Monitoring, 2022: 21(5): 2048.
    [29] YANG X, GAO Y, FANG C, et al. Deep learning-based bolt loosening detection for wind turbine towers [J]. Structural Control and Health Monitoring, 2022, 29(6): e2943.
    [30] ZHANG Y, YUEN K V. Bolt damage identification based on orientation-aware center point estimation network [J]. Structural Health Monitoring, 2022, 21(2): 438.
    [31] LI R, YUAN Y, ZHANG W, et al. Unified vision-based methodology for simultaneous concrete defect detection and geolocalization [J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(7): 527.
    [32] NI F T, ZHANG J, CHEN Z Q. Pixel-level crack delineation in images with convolutional feature fusion [J]. Structural Control and Health Monitoring, 2019, 26(1): e2286.
    [33] KALFARISI R, WU Z Y, SOH K. Crack detection and segmentation using deep learning with 3D reality mesh model for quantitative assessment and integrated visualization [J]. Journal of Computing in Civil Engineering, 2020, 34(3): 04020010.
    [34] ZHANG L, SHEN J, ZHU B. A research on an improved Unet-based concrete crack detection algorithm [J]. Structural Health Monitoring, 2021, 20(4): 1864.
    [35] FAN Z, LI C, CHEN Y, et al. Automatic crack detection on road pavements using encoder-decoder architecture [J]. Materials, 2020, 13(13): 2960.
    [36] LI Y, BAO T, XU B, et al. A deep residual neural network framework with transfer learning for concrete dams patch-level crack classification and weakly-supervised localization [J]. Measurement, 2022, 188: 110641.
    [37] JIANG Y, PANG D, LI C. A deep learning approach for fast detection and classification of concrete damage [J]. Automation in Construction, 2021, 128: 103785.
    [38] ZHANG A, WANG K C P, LI B, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network [J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805.
    [39] ZHANG A, WANG K C P, FEI Y, et al. Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet [J]. Journal of Computing in Civil Engineering, 2018, 32(5): 04018041.
    [40] JIANG Y, HAN S, LI D, et al. Automatic concrete sidewalk deficiency detection and mapping with deep learning [J]. Expert Systems with Applications, 2022, 207: 117980.
    [41] MCLAUGHLIN E, CHARRON N, NARASIMHAN S. Automated defect quantification in concrete bridges using robotics and deep learning [J]. Journal of Computing in Civil Engineering, 2020, 34(5): 04020029.
    [42] ISAILOVI? D, STOJANOVIC V, TRAPP M, et al. Bridge damage: Detection, IFC-based semantic enrichment and visualization [J]. Automation in Construction, 2020, 112: 103088.
    [43] YANG H, XU M, CHEN Y, et al. A postprocessing method based on regions and boundaries using convolutional neural networks and a new dataset for building extraction [J]. Remote Sensing, 2022, 14(3): 647.
    [44] JI M, LIU L, BUCHROITHNER M. Identifying collapsed buildings using post-earthquake satellite imagery and convolutional neural networks: A case study of the 2010 Haiti earthquake [J]. Remote Sensing, 2018, 10(11): 1689.
    [45] MA H, LIU Y, REN Y, et al. Detection of collapsed buildings in post-earthquake remote sensing images based on the improved YOLOv3 [J]. Remote Sensing, 2019, 12(1): 44.
    [46] WANG C, QIU X, HUAN H, et al. Earthquake-damaged buildings detection in very high-resolution remote sensing images based on object context and boundary enhanced loss [J]. Remote Sensing, 2021, 13(16): 3119.
    [47] MIURA H, ARIDOME T, MATSUOKA M. Deep learning-based identification of collapsed, non-collapsed and blue tarp-covered buildings from post-disaster aerial images [J]. Remote Sensing, 2020, 12(12): 1924.
    [48] DING J, ZHANG J, ZHAN Z, et al. A precision efficient method for collapsed building detection in post-earthquake UAV images based on the improved NMS algorithm and faster R-CNN [J]. Remote Sensing, 2022, 14(3): 663.
    [49] SHI L, ZHANG F, XIA J, et al. Identifying damaged buildings in aerial images using the object detection method [J]. Remote Sensing, 2021, 13(21): 4213.
    [50] ZHAN Y, LIU W, MARUYAMA Y. Damaged building extraction using modified mask R-CNN model using post-event aerial images of the 2016 Kumamoto earthquake [J]. Remote Sensing, 2022, 14(4): 1002.
    [51] CHENG C S, BEHZADAN A H, NOSHADRAVAN A. Deep learning for post-hurricane aerial damage assessment of buildings [J]. Computer-Aided Civil and Infrastructure Engineering, 2021, 36(6): 695.
    [52] RUDNER T G J, RU?WURM M, FIL J, et al. Multi3net: segmenting flooded buildings via fusion of multiresolution, multisensor, and multitemporal satellite imagery [C] // Proceedings of the AAAI Conference on Artificial Intelligence. [S.l.]: AAAI, 2019, 33(1): 702-709.
    [53] DUARTE D, NEX F, KERLE N, et al. Multi-resolution feature fusion for image classification of building damages with convolutional neural networks [J]. Remote Sensing, 2018, 10(10): 1636.
    [54] NEX F, DUARTE D, TONOLO F G, et al. Structural building damage detection with deep learning: Assessment of a state-of-the-art CNN in operational conditions [J]. Remote Sensing, 2019, 11(23): 2765.
    [55] ADRIANO B, YOKOYA N, XIA J, et al. Learning from multimodal and multitemporal earth observation data for building damage mapping [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175: 132.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

FANG Cheng, YU Shengxin, LI Yonggang, JIA Wanglong, YANG Pengbo, YANG Xinyue. Deep Learning-Based Computer Vision for Health Monitoring in Civil Engineering[J].同济大学学报(自然科学版),2024,52(2):213~222

Copy
Share
Article Metrics
  • Abstract:483
  • PDF: 610
  • HTML: 82
  • Cited by: 0
History
  • Received:August 24,2022
  • Online: February 27,2024
Article QR Code

Youare the 26819 visitor to this site

Copyright:JOURNAL OF TONGJI UNIVERSITY     Director:The Ministry of Education    The host:Tongji university.

Address:上海市四平路1239号同济大学内    Postcode:200092    TEL:86-21-65982344     E-mail:zrxb@tongji.edu.cn

Supported byBeijing E-Tiller Technology Development Co.,Ltd.