{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,23]],"date-time":"2024-07-23T09:10:40Z","timestamp":1721725840021},"reference-count":58,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T00:00:00Z","timestamp":1684368000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China Key Projects","award":["51938002"]},{"name":"National Natural Science Foundation of China","award":["52178029"]},{"name":"The Soft Science Project of the Ministry of Housing and Construction of China","award":["2020-R-022"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"As an essential material carrier of cultural heritage, the accurate identification and effective monitoring of buildings in traditional Chinese villages are of great significance to the sustainable development of villages. However, along with rapid urbanization in recent years, many towns have experienced problems such as private construction, hollowing out, and land abuse, destroying the traditional appearance of villages. This study combines deep learning technology and UAV remote sensing to propose a high-precision extraction method for conventional village architecture. Firstly, this study constructs the first sample database of traditional village architecture based on UAV remote sensing orthophotos of eight representative villages in Beijing, combined with fine classification; secondly, in the face of the diversity and complexity of the built environment in traditional villages, we use the Mask R-CNN instance segmentation model as the basis and Path Aggregate Feature Pyramid Network (PAFPN) and Atlas Space Pyramid Pool (ASPP) as the main strategies to enhance the backbone model for multi-scale feature extraction and fusion, using data increment and migration learning as auxiliary means to overcome the shortage of labeled data. The results showed that some categories could achieve more than 91% accuracy, with average precision, recall, F1-score, and Intersection over Union (IoU) values reaching 71.3% (+7.8%), 81.9% (+4.6%), 75.7% (+6.0%), and 69.4% (+8.5%), respectively. The application practice in Hexi village shows that the method has good generalization ability and robustness, and has good application prospects for future traditional village conservation.<\/jats:p>","DOI":"10.3390\/rs15102616","type":"journal-article","created":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T10:32:58Z","timestamp":1684405978000},"page":"2616","source":"Crossref","is-referenced-by-count":13,"title":["Traditional Village Building Extraction Based on Improved Mask R-CNN: A Case Study of Beijing, China"],"prefix":"10.3390","volume":"15","author":[{"given":"Wenke","family":"Wang","sequence":"first","affiliation":[{"name":"School of Architecture and Urban Planning, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"}]},{"given":"Yang","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Architecture and Urban Planning, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"},{"name":"Research Center for Urban Big Data Applications, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"}]},{"given":"Jie","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Architecture and Urban Planning, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"},{"name":"School of Architecture, Tsinghua University, Beijing 100084, China"}]},{"given":"Lujin","family":"Hu","sequence":"additional","affiliation":[{"name":"Research Center for Urban Big Data Applications, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"},{"name":"School of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"}]},{"given":"Shuo","family":"Li","sequence":"additional","affiliation":[{"name":"School of Architecture and Urban Planning, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"}]},{"given":"Ding","family":"He","sequence":"additional","affiliation":[{"name":"School of Architecture and Urban Planning, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"},{"name":"Research Center for Urban Big Data Applications, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"}]},{"given":"Fei","family":"Liu","sequence":"additional","affiliation":[{"name":"Research Center for Urban Big Data Applications, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"},{"name":"School of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, Beijing 100044, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,18]]},"reference":[{"key":"ref_1","first-page":"70","article-title":"Traditional folk art community and urban transformation: The case of the artists\u2019 village at Kalighat, India","volume":"36","author":"Ghosh","year":"2019","journal-title":"J. Archit. Plan. Res."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Xu, Q., and Wang, J. (2021). Recognition of values of traditional villages in Southwest China for sustainable development: A case study of Liufang village. Sustainability, 13.","DOI":"10.3390\/su13147569"},{"key":"ref_3","unstructured":"Liu, Y. (2018, January 23\u201324). On the protection dilemma of traditional Chinese villages: A case study of Xisuguazi Tibetan village. Proceedings of the Euro-Asian Conference on Corporate Social Responsibility (CSR) and Environmental Management\u2014Tourism, Society and Education Session (Part III), Tianjin, China."},{"key":"ref_4","unstructured":"Xie, X.B., and Li, X.J. (2019). The formation and transformation of \u201cCultural Matrix\u201d in traditional village. J. Hunan Univ. Soc. Sci. Ed., 33."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Liu, C., and Xu, M. (2021). Characteristics and influencing factors on the hollowing of traditional villages-taking 2645 villages from the Chinese traditional village catalogue (batch 5) as an example. Int. J. Environ. Res. Public Health, 18.","DOI":"10.3390\/ijerph182312759"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lu, Y., and Ahmad, Y. (2023). Heritage protection perspective of sustainable development of traditional villages in Guangxi, China. Sustainability, 15.","DOI":"10.3390\/su15043387"},{"key":"ref_7","first-page":"133","article-title":"The protection and tourism development path of ancient villages and old towns under the background of new-type urbanization: A case study of old town of Xuanzhou in Hunan province","volume":"38","author":"Liu","year":"2019","journal-title":"Geogr. Res."},{"key":"ref_8","unstructured":"Xia, M. (2018, January 12\u201313). The rural revitalization strategy-strategies for cultural heritage and transformation of traditional village Zhu Jiayu, Zhangqiu. Proceedings of the International Workshop on Advances in Social Sciences (IWASS), Hong Kong, China."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.foar.2012.12.004","article-title":"Landscape transition of historic villages in Southwest China","volume":"2","author":"Yu","year":"2013","journal-title":"Front. Archit. Res."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Olczak, B., Wilkosz-Mamcarczyk, M., Prus, B., Hodor, K., and Dixon-Gough, R. (2022). Application of the building cohesion method in spatial planning to shape patterns of the development in a suburban historical landscape of a \u2018village within Krak\u00f3w\u2019. Land Use Policy, 114.","DOI":"10.1016\/j.landusepol.2022.105997"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Fu, J., Zhou, J., and Deng, Y. (2021). Heritage values of ancient vernacular residences in traditional villages in Western Hunan, China: Spatial patterns and influencing factors. Build. Environ., 188.","DOI":"10.1016\/j.buildenv.2020.107473"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, N., Fang, M., Beauchamp, M., Jia, Z., and Zhou, Z. (2021). An indigenous knowledge-based sustainable landscape for mountain villages: The Jiabang rice terraces of Guizhou, China. Habitat Int., 111.","DOI":"10.1016\/j.habitatint.2021.102360"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Song, L.A., Gl, B., and Ming, X.C. (2020). The linguistic landscape in rural destinations: A case study of Hongcun Village in China. Tour. Manag., 77.","DOI":"10.1016\/j.tourman.2019.104005"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Xiong, Y., Chen, Q., Zhu, M., Zhang, Y., and Huang, K. (October, January 26). Accurate detection of historical buildings using aerial photographs and deep transfer learning. Proceedings of the IGARSS 2020\u20142020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA.","DOI":"10.1109\/IGARSS39084.2020.9323541"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5573","DOI":"10.1080\/01431161.2020.1734251","article-title":"A deep residual learning serial segmentation network for extracting buildings from remote sensing imagery","volume":"41","author":"Liu","year":"2020","journal-title":"Int. J. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1109\/LGRS.2020.2988326","article-title":"Building area estimation in drone aerial images based on mask R-CNN","volume":"18","author":"Chen","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, Y., Xu, W., Chen, H., Jiang, J., and Li, X. (2021). A novel framework based on mask R-CNN and histogram thresholding for scalable segmentation of new and old rural buildings. Remote Sens., 13.","DOI":"10.3390\/rs13061070"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wang, Y., Li, S., Teng, F., and Cai, H. (2022). Improved mask R-CNN for rural building roof type recognition from UAV high-resolution images: A case study in Hunan province, China. Remote Sens., 14.","DOI":"10.3390\/rs14020265"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhan, Y., Liu, W., and Maruyama, Y. (2022). Damaged building extraction using modified mask R-CNN model using post-event aerial images of the 2016 Kumamoto earthquake. Remote Sens., 14.","DOI":"10.3390\/rs14041002"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tejeswari, B., Sharma, S.K., Kumar, M.V.R., and Gupta, K. (2022). Building footprint extraction from space-borne imagery using deep neural networks. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., 641\u2013647.","DOI":"10.5194\/isprs-archives-XLIII-B2-2022-641-2022"},{"key":"ref_21","unstructured":"Katartzis, A., Sahli, H., Nyssen, E., and Cornelis, J. (2001, January 9\u201313). Detection of buildings from a single airborne image using a Markov random field model. Proceedings of the IEEE 2001 Geoscience and Remote Sensing Symposium (IGARSS), Sydney, NSW, Australia."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2386","DOI":"10.1109\/TGRS.2005.853570","article-title":"Rectangular building extraction from stereoscopic airborne Radar images","volume":"43","author":"Simonetto","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","unstructured":"Jung, C.R., and Schramm, R. (2004, January 20). Rectangle detection based on a windowed hough transform. Proceedings of the Brazilian Symposium on Computer Graphics & Image Processing, Curitiba, Brazil."},{"key":"ref_24","first-page":"394","article-title":"Feature fusion method for edge detection of color images","volume":"20","author":"Ma","year":"2009","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, Z.Z., and Zhang, Y.J. (2016). Building extraction from airborne laser point cloud using NDVI constrained watershed algorithm. Acta Opt. Sin., 36.","DOI":"10.3788\/AOS201636.1028002"},{"key":"ref_26","unstructured":"Zhou, S.L., Liang, D., Wang, H., and Kong, J. (2010). Remote sensing image segmentation approach based on quarter-tree and graph cut. Comput. Eng., 36."},{"key":"ref_27","unstructured":"Wei, D.Q. (2013). Research on Building Extraction Technology on High Resolution Remote Sensing Images. [Ph.D. Thesis, PLA Information Engineering University]."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Iyer, B., and Macleod, M.D. (2000, January 30). Multi-scale region segmentation of images using nonlinear methods. Proceedings of the Visual Communications and Image Processing, Perth, Australia.","DOI":"10.1117\/12.386610"},{"key":"ref_29","unstructured":"Zguira, A., Doggaz, N., and Zagrouba, E. (2011, January 5\u20137). Region-based objective evaluation of polygonal mesh segmentation methods. Proceedings of the VISAPP 2011\u2014Sixth International Conference on Computer Vision Theory and Applications, Algarve, Portugal."},{"key":"ref_30","unstructured":"Hui, Z., Fritts, J.E., and Goldman, S.A. (2005, January 18\u201320). A fast texture feature extraction method for region-based image segmentation. Proceedings of the SPIE\u2014The International Society for Optical Engineering, San Jose, CA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1109\/JSTARS.2011.2168195","article-title":"Morphological building\/shadow index for building extraction from high-resolution imagery over urban areas","volume":"5","author":"Huang","year":"2012","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"22034","DOI":"10.1109\/ACCESS.2018.2819705","article-title":"Building extraction from RGB VHR images using shifted shadow algorithm","volume":"6","author":"Gao","year":"2018","journal-title":"IEEE Access"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1016\/j.proeng.2011.07.069","article-title":"Use of digital surface model constructed from digital aerial images to detect collapsed buildings during earthquake","volume":"14","author":"Maruyama","year":"2011","journal-title":"Procedia Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1016\/j.isprsjprs.2010.02.002","article-title":"An efficient stochastic approach for building footprint extraction from digital elevation models","volume":"65","author":"Tournaire","year":"2010","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"185","DOI":"10.5623\/cig2017-401","article-title":"Building extraction from fused LiDAR and hyperspectral data using Random Forest Algorithm","volume":"71","author":"Saeid","year":"2017","journal-title":"Geomatica"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1109\/TGRS.2012.2205156","article-title":"Automatic detection and reconstruction of building radar footprints from single VHR SAR images","volume":"51","author":"Ferro","year":"2013","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1109\/TPAMI.2018.2844175","article-title":"Mask R-CNN","volume":"42","author":"He","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_38","unstructured":"Yuan, Y. Study on the Inheritance and Update of Residence in Jijiaying Village, Beijing University of Civil Engineering and Architecture."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs","volume":"40","author":"Chen","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","unstructured":"Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Jha, D., Smedsrud, P.H., Riegler, M.A., and Johansen, D. (2019). Simulamet. ResUNet++: An advanced architecture for medical image segmentation. arXiv.","DOI":"10.1109\/ISM46123.2019.00049"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Li, W., Sun, K., Zhao, H., Li, W., Wei, J., and Gao, S. (2022). Extracting buildings from high-resolution remote sensing images by deep ConvNets equipped with structural-cue-guided feature alignment. Int. J. Appl. Earth Obs. Geoinf., 113.","DOI":"10.1016\/j.jag.2022.102970"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017). Feature pyramid networks for object detection. IEEE Comput. Soc., 2117\u20132125.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18\u201323). Path aggregation network for instance segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00913"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Ge, J., Wang, C., Zhang, B., Xu, C., and Wen, X. (2022). Azimuth-sensitive object detection of high-resolution SAR images in complex scenes by using a spatial orientation attention enhancement network. Remote Sens., 14.","DOI":"10.3390\/rs14092198"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Yingying, X., Li, D., Xie, Q., Wu, Q., and Wang, J. (2021). Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN. Measurement, 178.","DOI":"10.1016\/j.measurement.2021.109316"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"He, D., Qiu, Y., Miao, J., Zou, Z., Li, K., Ren, C., and Shen, G. (2022). Improved mask R-CNN for obstacle detection of rail transit. Measurement, 190.","DOI":"10.1016\/j.measurement.2022.110728"},{"key":"ref_49","unstructured":"Fu, C.Y., Liu, W., Ranga, A., Tyagi, A., and Berg, A.C. (2017). DSSD: Deconvolutional Single Shot Detector. arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Kong, T., Yao, A., Chen, Y., and Sun, F. (2016, January 27\u201330). HyperNet: Towards accurate region proposal generation and joint object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.98"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Noh, H., Hong, S., and Han, B. (2016, January 27\u201330). Learning deconvolution network for semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, Las Vegas, NV, USA.","DOI":"10.1109\/ICCV.2015.178"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Wang, D.G., Lv, Q.Y., Wu, Y.F., and Fan, Z.Q. (2019). The characteristic of regional differentiation and impact mechanism of architecture style of traditional residence. J. Nat. Resour., 34.","DOI":"10.31497\/zrzyxb.20190906"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1016\/j.culher.2021.10.004","article-title":"Deep learning to detect built cultural heritage from satellite imagery\u2014Spatial distribution and size of vernacular houses in Sumba, Indonesia","volume":"52","author":"Monna","year":"2021","journal-title":"J. Cult. Herit."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Shorten, C., and Khoshgoftaar, T.M. (2019). A survey on image data augmentation for deep learning. J. Big Data, 6.","DOI":"10.1186\/s40537-019-0197-0"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2014, January 6\u201312). Spatial pyramid pooling in deep convolutional networks for visual recognition. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10578-9_23"},{"key":"ref_57","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2013MICCAI 2015: 18th International Conference, Munich, Germany, 5\u20139 October 2015, Springer International Publishing."},{"key":"ref_58","unstructured":"Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A.C., and Bengio, Y. (2014, January 8\u201313). Generative Adversarial Nets. Proceedings of the NIPS, Montreal, QC, Canada."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/10\/2616\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T10:46:44Z","timestamp":1684406804000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/10\/2616"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,18]]},"references-count":58,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["rs15102616"],"URL":"https:\/\/doi.org\/10.3390\/rs15102616","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,18]]}}}