{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,25]],"date-time":"2024-08-25T05:48:10Z","timestamp":1724564890500},"reference-count":47,"publisher":"Wiley","issue":"9","license":[{"start":{"date-parts":[[2019,7,18]],"date-time":"2019-07-18T00:00:00Z","timestamp":1563408000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Computer aided Civil Eng"],"published-print":{"date-parts":[[2019,9]]},"abstract":"Abstract<\/jats:title>Employing Deep Learning (DL) technologies to solve Civil Engineering problems is an emerging topic in recent years. However, due to the lack of labeled data, it is difficult to obtain accurate results with DL. One commonly used method to tackle this issue is to use affine transformation to augment the data set, but it can only generate new images that are highly correlated with the original ones. Moreover, unlike normal natural objects, distribution of structural images is much more complex and mixed. To address these challenges, Generative Adversarial Network (GAN) can be one feasible choice. We introduce one specific generative model, namely, Deep Convolutional Generative Adversarial Network (DCGAN) and propose a Leaf\u2010Bootstrapping (LB) method to improve the performance of this DCGAN. To effectively and quantitatively evaluate the quality of the synthetic images generated by DCGAN to complement human evaluation, Self\u2010Inception Score (SIS) and Generalization Ability (GA) are proposed. We also propose a pipeline based on Transfer Learning (TL) using synthetic images to help enhance a weak classifier performance under the condition of low\u2010data regime and limited computational resources. Finally, we conduct computer experiments with the proposed methods for two scenarios (scene level identification and damage state check) and one special synthetic data aggregation case. The results demonstrate the effectiveness and robustness of the proposed\u00a0methods.<\/jats:p>","DOI":"10.1111\/mice.12458","type":"journal-article","created":{"date-parts":[[2019,7,18]],"date-time":"2019-07-18T09:36:47Z","timestamp":1563442607000},"page":"755-773","update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":102,"title":["Deep leaf\u2010bootstrapping generative adversarial network for structural image data augmentation"],"prefix":"10.1111","volume":"34","author":[{"given":"Yuqing","family":"Gao","sequence":"first","affiliation":[{"name":"Department of Civil and Environmental Engineering University of California Berkeley CA USA"},{"name":"Tsinghua Berkeley Shenzhen Institute (TBSI) Shenzhen China"}]},{"given":"Boyuan","family":"Kong","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science University of California Berkeley CA USA"}]},{"given":"Khalid M.","family":"Mosalam","sequence":"additional","affiliation":[{"name":"Department of Civil and Environmental Engineering University of California Berkeley CA USA"},{"name":"Tsinghua Berkeley Shenzhen Institute (TBSI) Shenzhen China"},{"name":"Pacific Earthquake Engineering Research (PEER) Center Berkeley CA USA"}]}],"member":"311","published-online":{"date-parts":[[2019,7,18]]},"reference":[{"key":"e_1_2_8_2_1","unstructured":"Antoniou A. Storkey A. &Edwards H.(2017).Data augmentation generative adversarial networks.arXiv preprint arXiv:1711.04340."},{"key":"e_1_2_8_3_1","unstructured":"Arjovsky M. Chintala S. &Bottou L.(2017). Wasserstein GAN.arXiv preprint arXiv:1701.07875."},{"key":"e_1_2_8_4_1","unstructured":"Barratt S. &Sharma R.(2018). A note on the inception score.arXiv preprint arXiv:1801.01973."},{"key":"e_1_2_8_5_1","unstructured":"Berthelot D. Schumm T. &Metz L.(2017). BEGAN: Boundary equilibrium generative adversarial networks.arXiv preprint arXiv:1703.10717."},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12263"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12334"},{"key":"e_1_2_8_8_1","doi-asserted-by":"crossref","unstructured":"Deng J. Wei D. Richard S. Li\u2010Jia L. Kai L. &Fei\u2010Fei L.(2009).ImageNet: A large\u2010scale hierarchical image database.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 248\u2013255.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_8_9_1","doi-asserted-by":"crossref","unstructured":"Frid\u2010Adar M. Klang E. Amitai M. Goldberger J. &Greenspan H.(2018).Synthetic data augmentation using GAN for improved liver lesion classification.Biomedical Imaging (ISBI 2018). IEEE 289\u2013293.","DOI":"10.1109\/ISBI.2018.8363576"},{"key":"e_1_2_8_10_1","doi-asserted-by":"crossref","unstructured":"Gao Y. Li K. Mosalam K. &G\u00fcnay S.(2018).Deep residual network with transfer learning for image\u2010based structural damage recognition.Eleventh US National Conference on Earthquake Engineering Integrating Science Engineering & Policy Los Angeles CA.","DOI":"10.1111\/mice.12363"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12363"},{"key":"e_1_2_8_12_1","unstructured":"Goodfellow I.(2016).Nips 2016 tutorial: Generative adversarial networks.arXiv preprint arXiv:1701.00160."},{"key":"e_1_2_8_13_1","unstructured":"Goodfellow I. Pouget\u2010Abadie J. Mirza M. Xu B. Warde\u2010Farley D. Ozair S. \u2026Bengio Y.(2014).Generative adversarial nets.Advances in Neural Information Processing Systems 2672\u20132680."},{"key":"e_1_2_8_14_1","doi-asserted-by":"crossref","unstructured":"He K. Zhang X. Ren S. &Sun J.(2016).Deep residual learning for image recognition.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_8_15_1","unstructured":"Heusel M. Ramsauer H. Unterthiner T. Nessler B. &Hochreiter S.(2017).GANs trained by a two time\u2010scale update rule converge to a local Nash equilibrium.Advances in Neural Information Processing Systems 6626\u20136637."},{"key":"e_1_2_8_16_1","unstructured":"Ioffe S. &Szegedy C.(2015).Batch normalization: Accelerating deep network training by reducing internal covariate shift.arXiv preprint arXiv:1502.03167."},{"key":"e_1_2_8_17_1","doi-asserted-by":"crossref","unstructured":"Jia Y. Shelhamer E. Donahue J. Karayev S. Long J. Girshick R. \u2026Darrell T.(2014).Caffe: Convolutional architecture for fast feature embedding.Proceedings of the 22nd ACM International Conference on Multimedia. ACM 675\u2013678.","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_8_18_1","unstructured":"Jin Y. Zhang J. Li M. Tian Y. Zhu H. &Fang Z.(2017).Towards the automatic anime characters creation with generative adversarial networks.arXiv preprint arXiv:1708.05509."},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12375"},{"key":"e_1_2_8_20_1","unstructured":"Kingma D. P. &Ba J.(2014).Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980."},{"key":"e_1_2_8_21_1","unstructured":"Kitchen A. &Seah J.(2017).Deep generative adversarial neural networks for realistic prostate lesion MRI synthesis.arXiv preprint arXiv:1708.00129."},{"key":"e_1_2_8_22_1","volume-title":"Learning multiple layers of features from tiny images","author":"Krizhevsky A","year":"2009"},{"key":"e_1_2_8_23_1","doi-asserted-by":"crossref","unstructured":"LeCun Y. Bottou L. Bengio Y. &Haffner P.(1998).Gradient\u2010based learning applied to document recognition.Proceedings of the IEEE. IEEE 2278\u20132324.","DOI":"10.1109\/5.726791"},{"key":"e_1_2_8_24_1","doi-asserted-by":"crossref","unstructured":"Ledig C. Theis L. Huszr F. Caballero J. Cunningham A. Acosta A. \u2026Shi W.(2017).Photo\u2010realistic single image super\u2010resolution using a generative adversarial network.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 4681\u20134690.","DOI":"10.1109\/CVPR.2017.19"},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12313"},{"key":"e_1_2_8_26_1","doi-asserted-by":"crossref","unstructured":"Ma Y. &Ma M.(2017).A novel approach to artistic textual visualization via GAN.Proceedings of the 2017 International Conference on Information Technology. ACM 130\u2013135.","DOI":"10.1145\/3176653.3176722"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12387"},{"key":"e_1_2_8_28_1","unstructured":"Mariani G. Scheidegger F. Istrate R. Bekas C. &Malossi C.(2018).BAGAN: Data augmentation with balancing GAN.arXiv preprint arXiv:1803.09655."},{"key":"e_1_2_8_29_1","unstructured":"Mirza M. &Osindero S.(2014).Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784."},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12359"},{"key":"e_1_2_8_31_1","unstructured":"Nair V. &Hinton G. E.(2010).Rectified linear units improve restricted Boltzmann machines.Proceedings of the 27th International Conference on Machine Learning ICML 807\u2013814."},{"key":"e_1_2_8_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2017.05.029"},{"key":"e_1_2_8_33_1","unstructured":"Perez L. &Wang J.(2017).The effectiveness of data augmentation in image classification using deep learning.arXiv preprint arXiv:1712.04621."},{"key":"e_1_2_8_34_1","unstructured":"Radford A. Metz L. &Chintala S.(2015).Unsupervised representation learning with deep convolutional generative adversarial networks.arXiv preprint arXiv:1511.06434."},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1002\/tal.1400"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.engstruct.2017.10.070"},{"key":"e_1_2_8_37_1","doi-asserted-by":"publisher","DOI":"10.14359\/51689560"},{"key":"e_1_2_8_38_1","first-page":"2234","article-title":"Improved techniques for training GANs","author":"Salimans T.","year":"2016","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_8_39_1","unstructured":"Simonyan K. &Zisserman A.(2014).Very deep convolutional networks for large\u2010scale image recognition.arXiv preprint arXiv:1409.1556."},{"key":"e_1_2_8_40_1","doi-asserted-by":"crossref","unstructured":"Szegedy C. Vanhoucke V. Ioffe S. Shlens J. &Wojna Z.(2016).Rethinking the inception architecture for computer vision.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2818\u20132826.","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_2_8_41_1","unstructured":"Xu B. Wang N. Chen T. &Li M.(2015).Empirical evaluation of rectified activations in convolutional network.arXiv preprint arXiv:1505.00853."},{"key":"e_1_2_8_42_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12367"},{"key":"e_1_2_8_43_1","unstructured":"Yeum C. M. Dyke S. J. Ramirez J. &Benes B.(2016).Big visual data analysis for damage evaluation in civil engineering.Proceedings of the International Conference Smart Infrastructure & Construction Cambridge UK."},{"key":"e_1_2_8_44_1","unstructured":"Yu F. Seff A. Zhang Y. Song S. Funkhouser T. &Xiao J. (2015).LSUN: Construction of a large\u2010scale image dataset using deep learning with humans in the loop.arXiv preprint arXiv:1506.03365."},{"key":"e_1_2_8_45_1","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12297"},{"key":"e_1_2_8_46_1","first-page":"569","article-title":"Depth structure preserving scene image generation","volume":"1706","author":"Zhang W.","year":"2017","journal-title":"arXiv preprint arXiv"},{"key":"e_1_2_8_47_1","doi-asserted-by":"crossref","unstructured":"Zhang X. Wang Z. Liu D. &Ling Q.(2018).DADA: Deep adversarial data augmentation for extremely low data regime classification.arXiv preprint arXiv:1809.00981.","DOI":"10.1109\/ICASSP.2019.8683197"},{"key":"e_1_2_8_48_1","doi-asserted-by":"crossref","unstructured":"Zheng Z. Zheng L. &Yang Y.(2017).Unlabeled samples generated by GAN improve the person re\u2010identification baseline in vitro.arXiv preprint arXiv:1701.07717 3.","DOI":"10.1109\/ICCV.2017.405"}],"container-title":["Computer-Aided Civil and Infrastructure Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/mice.12458","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/mice.12458","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/mice.12458","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,10]],"date-time":"2023-09-10T02:05:24Z","timestamp":1694311524000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/mice.12458"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,18]]},"references-count":47,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2019,9]]}},"alternative-id":["10.1111\/mice.12458"],"URL":"https:\/\/doi.org\/10.1111\/mice.12458","archive":["Portico"],"relation":{},"ISSN":["1093-9687","1467-8667"],"issn-type":[{"value":"1093-9687","type":"print"},{"value":"1467-8667","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,18]]},"assertion":[{"value":"2019-07-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}