Abstract
Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component’s contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to automate these inspections through UAV-based image data collection and a software library for post-processing that helps in estimating the seismic structural parameters. The key parameters considered here are the distances between adjacent buildings, building plan-shape, building plan area, objects on the rooftop and rooftop layout. The accuracy of the proposed methodology in estimating the above-mentioned parameters is verified through field measurements taken using a distance measuring sensor and also from the data obtained through Google Earth. Additional details and code can be accessed from https://uvrsabi.github.io/.
K. Srivastava and D. Patel—denotes equal contribution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
UAV specification details can be found at the official DJI website: https://www.dji.com/mavic-mini.
- 2.
The ToF sensor can be found at: https://www.terabee.com/shop/lidar-tof-range-finders/teraranger-evo-60m/.
References
Spencer, B.F., Hoskere, V., Narazaki, Y.: Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering 5(2), 199–222 (2019). ISSN 2095-8099. https://doi.org/10.1016/j.eng.2018.11.030, https://www.sciencedirect.com/science/article/pii/S2095809918308130
Abdel-Qader, I., Abudayyeh, O., Kelly, M.: Analysis of edge-detection techniques for crack identification in bridges. J. Comput. Civ. Eng. 17, 10 (2003). https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)
Zhang, W., Zhang, Z., Qi, D., Liu, Y.: Automatic crack detection and classification method for subway tunnel safety monitoring. Sensors 14(10), 19307–19328 (2014). ISSN 1424-8220. https://doi.org/10.3390/s141019307, https://www.mdpi.com/1424-8220/14/10/19307
Liu, Y.-F., Cho, S., Spencer, B., Fan, J.-S.: Concrete crack assessment using digital image processing and 3D scene reconstruction. J. Comput. Civ. Eng. 30, 04014124 (2014). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000446
Adhikari, R.S., Moselhi, O., Bagchi, A.: A study of image-based element condition index for bridge inspection. In: ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction, vol. 30, p. 1. IAARC Publications (2013)
Paal, S., Jeon, J.-S., Brilakis, I., Desroches, R.: Automated damage index estimation of reinforced concrete columns for post-earthquake evaluations. J. Struct. Eng. 141, 04014228 (2015). https://doi.org/10.1061/(ASCE)ST.1943-541X.0001200
Yeum, C.M., Dyke, S.J.: Vision-based automated crack detection for bridge inspection. Comput.-Aided Civ. Infrastruct. Eng. 30(10), 759–770 (2015)
Jahanshahi, M.R., Chen, F.-C., Joffe, C., Masri, S.F.: Vision-based quantitative assessment of microcracks on reactor internal components of nuclear power plants. Struct. Infrastruct. Eng. 13(8), 1013–1026 (2017)
Son, H., Hwang, N., Kim, C., Kim, C.: Rapid and automated determination of rusted surface areas of a steel bridge for robotic maintenance systems. Autom. Constr. 42, 13–24 (2014)
Shen, H.-K., Chen, P.-H., Chang, L.-M.: Automated steel bridge coating rust defect recognition method based on color and texture feature. Autom. Constr. 31, 338–356 (2013)
Medeiros, F.N.S., Ramalho, G.L.B., Bento, M.P., Medeiros, L.C.L.: On the evaluation of texture and color features for nondestructive corrosion detection. EURASIP J. Adv. Signal Process. 2010, 1–7 (2010)
Cha, Y.-J., Choi, W., Büyüköztürk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civ. Infrastruct. Eng. 32(5), 361–378 (2017)
Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detection using deep convolutional neural network. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3708–3712. IEEE (2016)
Atha, D.J., Jahanshahi, M.R.: Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection. Struct. Health Monit. 17(5), 1110–1128 (2018)
Yeum, C.M., Dyke, S.J., Ramirez, J.: Visual data classification in post-event building reconnaissance. Eng. Struct. 155, 16–24 (2018)
Cha, Y.-J., Choi, W., Suh, G., Mahmoudkhani, S., Büyüköztürk, O.: Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput.-Aided Civ. Infrastruct. Eng. 33(9), 731–747 (2018)
Zhang, A., et al.: Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Comput.-Aided Civ. Infrastruct. Eng. 32(10), 805–819 (2017)
Hoskere, V., Narazaki, Y., Hoang, T., Spencer Jr., B.F.: Vision-based structural inspection using multiscale deep convolutional neural networks. arXiv preprint arXiv:1805.01055 (2018)
Hoskere, V., Narazaki, Y., Hoang, T.A., Spencer Jr., B.F.: Towards automated post-earthquake inspections with deep learning-based condition-aware models. arXiv preprint arXiv:1809.09195 (2018)
Zhu, Z., Brilakis, I.: Concrete column recognition in images and videos. J. Comput. Civ. Eng. 24(6), 478–487 (2010)
Koch, C., Paal, S.G., Rashidi, A., Zhu, Z., König, M., Brilakis, I.: Achievements and challenges in machine vision-based inspection of large concrete structures. Adv. Struct. Eng. 17(3), 303–318 (2014)
Xiong, X., Adan, A., Akinci, B., Huber, D.: Automatic creation of semantically rich 3d building models from laser scanner data. Autom. Constr. 31, 325–337 (2013)
Armeni, I., et al.: 3D semantic parsing of large-scale indoor spaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1534–1543 (2016)
Golparvar-Fard, M., Bohn, J., Teizer, J., Savarese, S., Peña-Mora, F.: Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques. Autom. Constr. 20(8), 1143–1155 (2011)
Lu, R., Brilakis, I., Middleton, C.R.: Detection of structural components in point clouds of existing RC bridges. Comput.-Aided Civ. Infrastruct. Eng. 34(3), 191–212 (2019)
Gao, Y., Mosalam, K.M.: Deep transfer learning for image-based structural damage recognition. Comput.-Aided Civ. Infrastruct. Eng. 33(9), 748–768 (2018)
Liang, X.: Image-based post-disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization. Comput.-Aided Civ. Infrastruct. Eng. 34(5), 415–430 (2019)
Yeum, C.M., Choi, J., Dyke, S.J.: Automated region-of-interest localization and classification for vision-based visual assessment of civil infrastructure. Struct. Health Monit. 18(3), 675–689 (2019)
Narazaki, Y., Hoskere, V., Hoang, T.A., Fujino, Y., Sakurai, A., Spencer Jr., B.F.: Vision-based automated bridge component recognition with high-level scene consistency. Comput.-Aided Civ. Infrastruct. Eng. 35(5), 465–482 (2020)
Dimitrov, A., Golparvar-Fard, M.: Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections. Adv. Eng. Inform. 28(1), 37–49 (2014)
Golparvar-Fard, M., Pena-Mora, F., Savarese, S.: Automated progress monitoring using unordered daily construction photographs and IFC-based building information models. J. Comput. Civ. Eng. 29(1), 04014025 (2015)
Hamledari, H., Davari, S., Azar, R., McCabe, B., Flager, F., Fischer, M.: UAV-enabled site-to-BIM automation: aerial robotic-and computer vision-based development of as-built/as-is BIMs and quality control. In: Construction research congress, pp. 336–346 (2017)
Pelizari, P.A., Geiß, C., Aguirre, P., María, H.S., Peña, Y.M., Taubenböck, H.: Automated building characterization for seismic risk assessment using street-level imagery and deep learning. ISPRS J. Photogram. Remote Sens. 180, 370–386 (2021). ISSN 0924-2716. https://doi.org/10.1016/j.isprsjprs.2021.07.004, https://www.sciencedirect.com/science/article/pii/S0924271621001817
Gonzalez, D., et al.: Automatic detection of building typology using deep learning methods on street level images. Build. Environ. 177, 106805 (2020). ISSN 0360-1323. https://doi.org/10.1016/j.buildenv.2020.106805, https://www.sciencedirect.com/science/article/pii/S0360132320301633
Hackl, J., Adey, B., Woźniak, M., Schümperlin, O.: Use of unmanned aerial vehicle photogrammetry to obtain topographical information to improve bridge risk assessment. J. Infrastruct. Syst. 24, 04017041 (2018). https://doi.org/10.1061/(ASCE)IS.1943-555X.0000393
Miari, M., Choong, K.K., Jankowski, R.: Seismic pounding between adjacent buildings: identification of parameters, soil interaction issues and mitigation measures. Soil Dyn. Earthq. Eng. 121, 135–150 (2019). ISSN 0267-7261. https://doi.org/10.1016/j.soildyn.2019.02.024, https://www.sciencedirect.com/science/article/pii/S0267726118313848
Carboney, J.A., García, H.J., Ortega, R., Iglesias, J.: The Mexico earthquake of September 19, 1985 - statistics of damage and of retrofitting techniques in reinforced concrete buildings affected by the 1985 earthquake. Earthq. Spectra 5 (1989). https://doi.org/10.1193/1.1585516
Kasai, K., Maison, B.F.: Building pounding damage during the 1989 Loma Prieta earthquake. Eng. Struct. 19(3): 195–207 (1997). ISSN 0141-0296. https://doi.org/10.1016/S0141-0296(96)00082-X, https://www.sciencedirect.com/science/article/pii/S014102969600082X
Lin, J.-H., Weng, C.-C.: A study on seismic pounding probability of buildings in Taipei metropolitan area. J. Chin. Inst. Eng. 25(2), 123–135 (2002). https://doi.org/10.1080/02533839.2002.9670687, https://doi.org/10.1080/02533839.2002.9670687
Kaushik, H.B., Da, K., Sahoo, D.R., Kharel, G.: Performance of structures during the Sikkim earthquake of 14 February 2006. Curr. Sci. 91, 449–455 (2006)
Bektaş, N., Kegyes-Brassai, O.: Conventional RVS methods for seismic risk assessment for estimating the current situation of existing buildings: a state-of-the-art review. Sustainability 14(5) (2022). ISSN 2071-1050. https://doi.org/10.3390/su14052583, https://www.mdpi.com/2071-1050/14/5/2583
Ramancharla, P., et al.: A primer on rapid visual screening (RVS) consolidating earthquake safety assessment efforts in India (2020)
Vacca, G., Dessì, A., Sacco., A.: The use of nadir and oblique UAV images for building knowledge. ISPRS Int. J. Geo-Inf. 6, 393 (2017). https://doi.org/10.3390/ijgi6120393
Arnold, C., Reitherman, R.: Building Configuration and Seismic Design. Wiley, Hoboken (1982)
Sahar, L., Muthukumar, S., French, S.P.: Using aerial imagery and GIS in automated building footprint extraction and shape recognition for earthquake risk assessment of urban inventories. IEEE Trans. Geosci. Remote Sens. 48(9), 3511–3520 (2010)
Chen, D., Zhang, L., Li, J., Liu, R.: Urban building roof segmentation from airborne lidar point clouds. Int. J. Remote Sens. 33(20), 6497–6515 (2012). https://doi.org/10.1080/01431161.2012.690083
Vo, A.-V., Truong-Hong, L., Laefer, D.F., Bertolotto, M.: Octree-based region growing for point cloud segmentation. ISPRS J. Photogram. Remote Sens. 104, 88–100 (2015). ISSN 0924-2716. https://doi.org/10.1016/j.isprsjprs.2015.01.011, https://www.sciencedirect.com/science/article/pii/S0924271615000283
Sampath, A., Shan, J.: Segmentation and reconstruction of polyhedral building roofs from aerial lidar point clouds. IEEE Trans. Geosci. Remote Sens. 48(3), 1554–1567 (2010). https://doi.org/10.1109/TGRS.2009.2030180
Dong, Z., Yang, B., Hu, P., Scherer, S.: An efficient global energy optimization approach for robust 3D plane segmentation of point clouds. ISPRS J. Photogram. Remote Sens. 137, 112–133 (2018). ISSN 0924-2716. https://doi.org/10.1016/j.isprsjprs.2018.01.013, https://www.sciencedirect.com/science/article/pii/S0924271618300133
Li, W., He, C., Fang, J., Zheng, J., Fu, H., Yu, L.: Semantic segmentation-based building footprint extraction using very high-resolution satellite images and multi-source GIS data. Remote Sens. 11(4) (2019). ISSN 2072-4292. https://doi.org/10.3390/rs11040403, https://www.mdpi.com/2072-4292/11/4/403
Qin, Y., Wu, Y., Li, B., Gao, S., Liu, M., Zhan, Y.: Semantic segmentation of building roof in dense urban environment with deep convolutional neural network: a case study using GF2 VHR imagery in China. Sensors 19, 1164 (2019). https://doi.org/10.3390/s19051164
Bhat, A.S., Shivaprakash, A.V., Prasad, N.S., Nagaraj, C.: Template matching technique for panoramic image stitching. In: 2013 7th Asia Modelling Symposium, pp. 111–115 (2013). https://doi.org/10.1109/AMS.2013.22
Adwan, S., Alsaleh, I., Majed, R.: A new approach for image stitching technique using dynamic time warping (DTW) algorithm towards scoliosis X-ray diagnosis. Measurements 84, 32–46 (2016). https://doi.org/10.1016/j.measurement.2016.01.039
Bonny, M., Uddin, M.: A technique for panorama-creation using multiple images. Int. J. Adv. Comput. Sci. Appl. 11 (2020). https://doi.org/10.14569/IJACSA.2020.0110293
Alomran, M., Chai, D.: Feature-based panoramic image stitching. In: 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1–6 (2016). https://doi.org/10.1109/ICARCV.2016.7838721
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004). ISSN 1573-1405. https://doi.org/10.1023/B:VISI.0000029664.99615.94
Zhang, Y., Yang, L., Wang, Z.: Research on video image stitching technology based on surf. In: 2012 Fifth International Symposium on Computational Intelligence and Design, vol. 2, pp. 335–338 (2012). https://doi.org/10.1109/ISCID.2012.235
Tsao, P., Ik, T.-U., Chen, G.-W., Peng, W.-C.: Stitching aerial images for vehicle positioning and tracking. In: 2018 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 616–623. IEEE (2018)
Liu, Y., Xue, F., Huang, H.: UrbanScene3D: a large scale urban scene dataset and simulator (2021)
Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., Girshick, R.: Detectron2 (2019). https://github.com/facebookresearch/detectron2
Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_31
Wang, Y., et al.: LEDNet: a lightweight encoder-decoder network for real-time semantic segmentation. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1860–1864 (2019). https://doi.org/10.1109/ICIP.2019.8803154
Zhou, X., Girdhar, R., Joulin, A., Krähenbühl, P., Misra, I.: Detecting twenty-thousand classes using image-level supervision. arXiv preprint arXiv:2201.02605 (2021)
Acknowledgement
The authors acknowledge the financial support provided by IHUB, IIIT Hyderabad to carry out this research work under the project: IIIT-H/IHub/Project/Mobility/2021-22/M2-003.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Srivastava, K. et al. (2023). UAV-Based Visual Remote Sensing for Automated Building Inspection. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13807. Springer, Cham. https://doi.org/10.1007/978-3-031-25082-8_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-25082-8_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25081-1
Online ISBN: 978-3-031-25082-8
eBook Packages: Computer ScienceComputer Science (R0)