{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T23:14:35Z","timestamp":1725578075858},"reference-count":48,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2022,9,21]],"date-time":"2022-09-21T00:00:00Z","timestamp":1663718400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100000140","name":"U.S. Department of Transportation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000140","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Journal of Field Robotics"],"published-print":{"date-parts":[[2023,1]]},"abstract":"Abstract<\/jats:title>Human\u2010made concrete structures require cutting\u2010edge inspection tools to ensure the quality of the construction to meet the applicable building codes and to maintain the sustainability of the aging infrastructure. This paper introduces a wall\u2010climbing robot for metric concrete inspection that can reach difficult\u2010to\u2010access locations with a close\u2010up view for visual data collection and real\u2010time flaws detection and localization. The wall\u2010climbing robot is able to detect concrete surface flaws (i.e., cracks and spalls) and produce a defect\u2010highlighted 3D model with extracted location clues and metric measurements. The system encompasses four modules, including a data collection module to capture RGB\u2010D frames and inertial measurement unit data, a visual\u2013inertial navigation system module to generate pose\u2010coupled keyframes, a deep neural network module (namely InspectionNet) to classify each pixel into three classes (background, crack, and spall), and a semantic reconstruction module to integrate per\u2010frame measurement into a global volumetric model with defects highlighted. We found that commercial RGB\u2010D camera output depth is noisy with holes, and a Gussian\u2010Bilateral filter for depth completion is introduced to inpaint the depth image. The method achieves the state\u2010of\u2010the\u2010art depth completion accuracy even with large holes. Based on the semantic mesh, we introduce a coherent defect metric evaluation approach to compute the metric measurement of crack and spall area (e.g., length, width, area, and depth). Field experiments on a concrete bridge demonstrate that our wall\u2010climbing robot is able to operate on a rough surface and can cross over shallow gaps. The robot is capable to detect and measure surface flaws under low illuminated environments and texture\u2010less environments. Besides the robot system, we create the first publicly accessible concrete structure spalls and cracks data set that includes 820 labeled images and over 10,000 field\u2010collected images and release it to the research community.<\/jats:p>","DOI":"10.1002\/rob.22119","type":"journal-article","created":{"date-parts":[[2022,9,21]],"date-time":"2022-09-21T12:55:57Z","timestamp":1663764957000},"page":"110-129","update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Automated wall\u2010climbing robot for concrete construction inspection"],"prefix":"10.1002","volume":"40","author":[{"given":"Liang","family":"Yang","sequence":"first","affiliation":[{"name":"The CCNY Robotics Lab, Electrical Engineering Department The City College of New York New York New York USA"}]},{"given":"Bing","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Automotive Engineering Clemson University Clemson South Carolina USA"}]},{"given":"Jinglun","family":"Feng","sequence":"additional","affiliation":[{"name":"The CCNY Robotics Lab, Electrical Engineering Department The City College of New York New York New York USA"}]},{"given":"Guoyong","family":"Yang","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences University of Chinese Academy of Sciences Liaoning China"}]},{"given":"Yong","family":"Chang","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences University of Chinese Academy of Sciences Liaoning China"}]},{"given":"Biao","family":"Jiang","sequence":"additional","affiliation":[{"name":"Department of Natural Sciences Hostos Community College New York New York USA"}]},{"given":"Jizhong","family":"Xiao","sequence":"additional","affiliation":[{"name":"The CCNY Robotics Lab, Electrical Engineering Department The City College of New York New York New York USA"}]}],"member":"311","published-online":{"date-parts":[[2022,9,21]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.161"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364907079283"},{"key":"e_1_2_9_4_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00432"},{"key":"e_1_2_9_5_1","doi-asserted-by":"crossref","unstructured":"Bo\u017ei\u0107 M. Jerbi\u0107 B.&\u0160vaco M.(2021) Development of a mobile wall\u2010climbing robot with a hybrid adhesion system. In:2021 44th international convention on information communication and electronic technology (MIPRO) IEEE pp.1136\u20131142.","DOI":"10.23919\/MIPRO52101.2021.9596717"},{"key":"e_1_2_9_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/MED.2016.7535885"},{"key":"e_1_2_9_7_1","doi-asserted-by":"publisher","DOI":"10.21660\/2018.51.35376"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12564"},{"key":"e_1_2_9_9_1","doi-asserted-by":"crossref","unstructured":"Deng J.&Han Y.(2013)A real\u2010time system of lane detection and tracking based on optimized ransac b\u2010spline fitting. In:Proceedings of the 2013 research in adaptive and convergent systems pp.157\u2013164.","DOI":"10.1145\/2513228.2513280"},{"issue":"3","key":"e_1_2_9_10_1","first-page":"33","article-title":"Developments in wall climbing robots: a review","volume":"2","author":"Dethe R. D","year":"2014","journal-title":"International Journal of Engineering Research and General Science"},{"key":"e_1_2_9_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICARCV.2016.7838682"},{"key":"e_1_2_9_12_1","unstructured":"Everingham M.&Winn J.(2011)The pascal visual object classes challenge 2012 (voc2012) development kit. Pattern analysis Statistical Modelling and Computational Learning Technical Report."},{"issue":"3","key":"e_1_2_9_13_1","first-page":"188","article-title":"An autonomous wall\u2010climbing robot for inspection of reinforced concrete structures: sircaur","volume":"1","author":"Garrido G. G.","year":"2021","journal-title":"Journal of Artificial Intelligence and Technology"},{"key":"e_1_2_9_14_1","doi-asserted-by":"publisher","DOI":"10.2355\/isijinternational.ISIJINT-2015-041"},{"key":"e_1_2_9_15_1","doi-asserted-by":"crossref","unstructured":"Hallermann N.&Morgenthal G.(2014)Visual inspection strategies for large bridges using unmanned aerial vehicles (UAV). In:Proceedings of 7th IABMAS international conference on bridge maintenance safety and management pp.661\u2013667.","DOI":"10.1201\/b17063-96"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1061\/(ASCE)CP.1943-5487.0000890"},{"key":"e_1_2_9_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/stc.2381"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.aei.2015.01.008"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21725"},{"key":"e_1_2_9_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2018.2791430"},{"issue":"2","key":"e_1_2_9_21_1","first-page":"1\u201012","article-title":"Image\u2010based concrete crack detection using convolutional neural network and exhaustive search technique","volume":"2019","author":"Li S.","year":"2019","journal-title":"Advances in Civil Engineering"},{"key":"e_1_2_9_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/37402.37422"},{"key":"e_1_2_9_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2001.937655"},{"key":"e_1_2_9_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.autcon.2006.12.010"},{"key":"e_1_2_9_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2007.364024"},{"key":"e_1_2_9_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2017.2705103"},{"key":"e_1_2_9_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"e_1_2_9_28_1","unstructured":"Office of Construction New York. (2021) Construction inspection manual.https:\/\/www.dot.ny.gov\/main\/business-center\/contractors\/construction-division\/forms-manuals-computer-applications-general-information\/CIM Part 1B:1\u2013349."},{"key":"e_1_2_9_29_1","unstructured":"Paszke A. Gross S. Chintala S.&Chanan G.(2017) Pytorch: an imperative style high\u2010performance deep learning library.33rd Conference on Advances in Neural Information Processing Systems (NeurIPS 2019) pp. 32."},{"key":"e_1_2_9_30_1","volume-title":"Digital video and HD: algorithms and Interfaces","author":"Poynton C.","year":"2012"},{"key":"e_1_2_9_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2014.2354314"},{"key":"e_1_2_9_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2014.2354314"},{"key":"e_1_2_9_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2012.06.003"},{"key":"e_1_2_9_34_1","doi-asserted-by":"crossref","unstructured":"Ronneberger O. Fischer P.&Brox T.(2015a) U\u2010net: convolutional networks for biomedical image segmentation 234\u2013241.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_9_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_9_36_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2007.01016.x"},{"key":"e_1_2_9_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/INDICON.2014.7030589"},{"key":"e_1_2_9_38_1","unstructured":"Simonyan K.&Zisserman A.(2015)Very deep convolutional networks for large\u2010scale image recognition.3rd International Conference on Learning Representations (ICLR) pp.1\u201314."},{"key":"e_1_2_9_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.autcon.2005.02.006"},{"key":"e_1_2_9_40_1","unstructured":"US Department of Transportation Federal Highway Administration.(2021) Construction program management and inspection guide 1\u20134.https:\/\/highways.dot.gov\/federal-lands\/construction\/forms-wfl-internal\/section-552"},{"key":"e_1_2_9_41_1","doi-asserted-by":"publisher","DOI":"10.1061\/(ASCE)CP.1943-5487.0000451"},{"key":"e_1_2_9_42_1","doi-asserted-by":"crossref","unstructured":"Xie S.&Tu Z.(2015)Holistically\u2010nested edge detection. In:Proceedings of the IEEE International Conference on Computer Vision pp.1395\u20131403.","DOI":"10.1109\/ICCV.2015.164"},{"key":"e_1_2_9_43_1","doi-asserted-by":"crossref","unstructured":"Yang L. Li B. Li W. Jiang B.&Xiao J.(2018)Semantic metric 3d reconstruction for concrete inspection. In:Proceedings of the IEEE conference on computer vision and pattern recognition workshops pp.1543\u20131551.","DOI":"10.1109\/CVPRW.2018.00204"},{"key":"e_1_2_9_44_1","volume-title":"IEEE\/RSJ international conference on intelligent robots and systems","author":"Yang L.","year":"2017"},{"key":"e_1_2_9_45_1","doi-asserted-by":"crossref","unstructured":"Yang L. Li B. Yang G. Chang Y. Liu Z. Jiang B.&Xiao J.(2019)Deep neural network based visual inspection with 3d metric measurement of concrete defects using wall\u2010climbing robot. In:IEEE\/RSJ international conference on intelligent robots and systems pp.2849\u20132854.","DOI":"10.1109\/IROS40897.2019.8968195"},{"key":"e_1_2_9_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.autcon.2006.05.003"},{"key":"e_1_2_9_47_1","doi-asserted-by":"publisher","DOI":"10.1177\/0361198118782025"},{"key":"e_1_2_9_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.888718"},{"key":"e_1_2_9_49_1","unstructured":"Zhou Q.\u2010Y. Park J.&Koltun V.(2018)Open3D: a modern library for 3D data processing. Online (arXiv:1801.09847)."}],"container-title":["Journal of Field Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/rob.22119","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/rob.22119","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/rob.22119","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,19]],"date-time":"2023-08-19T22:42:29Z","timestamp":1692484949000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rob.22119"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,21]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1]]}},"alternative-id":["10.1002\/rob.22119"],"URL":"https:\/\/doi.org\/10.1002\/rob.22119","archive":["Portico"],"relation":{},"ISSN":["1556-4959","1556-4967"],"issn-type":[{"value":"1556-4959","type":"print"},{"value":"1556-4967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,21]]},"assertion":[{"value":"2022-02-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-08-29","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}