Abstract
Manual surface inspection methods performed by quality inspectors do not satisfy the continuously increasing quality standards of industrial manufacturing processes. Machine vision provides a solution by using an automated visual inspection (AVI) system to perform quality inspection and remove defective products. Numerous studies and works have been conducted on surface inspection algorithms. With the advent of deep learning, a number of new algorithms have been developed for better inspection. In this paper, the state-of-the-art in surface defect inspection using deep learning is presented. In particular, we focus on the inspection of industrial products in semiconductor, steel, and fabric manufacturing processes. This work makes three contributions. First, we present the prior literature reviews on vision-based surface defect inspection and analyze the recent AVI-related hardware and software. Second, we review traditional surface defect inspection algorithms including statistical methods, spectral methods, model-based methods, and learning-based methods. Third, we investigate recent advances in deep learning-based inspection algorithms and present their applications in the steel, fabric, and semiconductor industries. Furthermore, we provide information on publicly available datasets containing surface image samples to facilitate the research on deep learning-based surface inspection.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00170-021-06592-8/MediaObjects/170_2021_6592_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00170-021-06592-8/MediaObjects/170_2021_6592_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00170-021-06592-8/MediaObjects/170_2021_6592_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00170-021-06592-8/MediaObjects/170_2021_6592_Fig4_HTML.png)
Similar content being viewed by others
Data availability
Not applicable.
References
Steger C, Ulrich M, Wiedemann C (2018) Machine vision algorithms and applications: second completely revised and Enlarged Edition. Wiley-VCH, Hoboken
Hornberg A (2017) Handbook of machine and computer vision: the guide for developers and users, Second edn. Wiley-VCH. https://doi.org/10.1002/9783527413409
Sun XH, Gu JA, Tang SX, Li J (2018) Research progress of visual inspection technology of steel products-a review. Appl Sci-Basel 8(11). https://doi.org/10.3390/app8112195
Golnabi H, Asadpour A (2007) Design and application of industrial machine vision systems. Robot Comput Integr Manuf 23(6):630–637. https://doi.org/10.1016/j.rcim.2007.02.005
Ozseven T (2019) Surface defect detection and quantification with image processing methods. In: Ozseven T (ed) Theoretical investigations and applied studies in engineering. Ekin Publishing House, pp 63–98
Newman TS, Jain AK (1995) A survey of automated visual inspection. Comput Vis Image Underst 61(2):231–262
Neogi N, Mohanta DK, Dutta PK (2014) Review of vision-based steel surface inspection systems. EURASIP J Image Vide:1–19. https://doi.org/10.1186/1687-5281-2014-50
Gao C, Zhou J, Wong WK, Gao T Woven fabric defect detection based on convolutional neural network for binary classification. In: Artificial Intelligence on Fashion and Textiles Conference, AIFT 2018, June 27, 2018 - June 29, 2018, Hong Kong, China, 2019. Advances in intelligent systems and computing. Springer Verlag, pp 307–313. https://doi.org/10.1007/978-3-319-99695-0_37
Huang SH, Pan YC (2015) Automated visual inspection in the semiconductor industry: a survey. Comput Ind 66:1–10
Malamas EN, Petrakis EGM, Zervakis M, Petit L, Legat JD (2003) A survey on industrial vision systems, applications and tools. Image Vis Comput 21(2):171–188. https://doi.org/10.1016/S0262-8856(02)00152-X
Xie X (2008) A review of recent advances in surface defect detection using texture analysis techniques. Electron Lett Comput Vis Image Anal 7(3):1–22
Kumar A (2008) Computer-vision-based fabric defect detection: a survey. IEEE Trans Ind Electron 55(1):348–363. https://doi.org/10.1109/Tie.2007.896476
Mahajan PM, Kolhe SR, Patil PM (2009) A review of automatic fabric defect detection techniques. Adv Comput Res 1(2):18–29
Hani AFM, Malik AS, Kamil R, Thong CM (2012) A review of SMD-PCB defects and detection algorithms. Proc SPIE 8350. https://doi.org/10.1117/12.920531
Ngan HYT, Pang GKH, Yung NHC (2011) Automated fabric defect detection-a review. Image Vis Comput 29(7):442–458. https://doi.org/10.1016/j.imavis.2011.02.002
Hanbay K, Talu MF, Ozguven OF (2016) Fabric defect detection systems and methods-a systematic literature review. Optik 127(24):11960–11973. https://doi.org/10.1016/j.ijleo.2016.09.110
Anitha DB, Rao M (2017) A survey on defect detection in bare PCB and assembled PCB using image processing techniques. In: 2017 2nd Ieee international conference on wireless communications, signal processing and networking (Wispnet), pp 39–43. https://doi.org/10.1109/WiSPNET.2017.8299715
Lu R, Wu A, Zhang T, Wang Y (2018) Review on automated optical (visual) inspection and its application in defect detection. Acta Opt Sin 38(437 (8)):15–50
Shirvaikar M (2006) Trends in automated visual inspection. J Real Time Image Process 1(1):41–43. https://doi.org/10.1007/s11554-006-0009-6
Shreya SR, Priya CS, Rajeshware GS (2017) Design of machine vision system for high speed manufacturing environments. In: India Conference, 2017
OpenCV Tutorials. https://docs.opencv.org/master/d9/df8/tutorial_root.html. Accessed Oct. 2019
HALCON_18.11_brochure. https://www.mvtec.com. Accessed Oct. 2019
VisionPro. https://www.cognex.com. Accessed Oct. 2019
Demant C, Streicher-Abel B, Garnica C (2013) Industrial image processing: visual quality control in manufacturing, 2nd edn. Springer. https://doi.org/10.1007/978-3-642-33905-9
Van Gool L, Wambacq P, Oosterlinck A (1991) Intelligent robotic vision systems. Marcel Dekker Inc, New York
Bible RE (1984) Automated optical inspection of printed circuit boards. Test Meas World Oct.:208–213
Moganti M, Ercal F, Dagli CH, Shou T (1996) Automatic PCB inspection algorithms: a survey. Comput Vis Image Underst 63(2):287–313
Silven O, Virtanen I, Pietikainen M (1985) Cad data-based comparison method for printed wiring board (PWB) inspection. In: Society of Photo-optical Instrumentation Engineers Conference Series, 17 January 1985. https://doi.org/10.1117/12.946210
Li YD, Zhao WG, Pan JH (2017) Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Trans Autom Sci Eng 14(2):1256–1264. https://doi.org/10.1109/Tase.2016.2520955
Liu K, Wang H, Chen H, Qu E, Sun H (2017) Steel surface defect detection using a new Haar-Weibull-variance model in unsupervised manner. IEEE Trans Instrum Meas 99:1–12
Huangpeng Q, Zhang H, Zeng XR, Huang WW (2018) Automatic visual defect detection using texture prior and low-rank representation. IEEE Access 6:37965–37976. https://doi.org/10.1109/Access.2018.2852663
Haralick RM, Shanmugam K, Dinstein I’H (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC-3(6):610–621. https://doi.org/10.1109/TSMC.1973.4309314
Ojala T, Harwood I (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recogn 29(1):51–59
Tajeripour F, Kabir E, Sheikhi A (2008) Fabric defect detection using modified local binary patterns. EURASIP J Adv Sig Process 2008. https://doi.org/10.1155/2008/783898
Tang B, Kong J, Wu S (2017) Review of surface defect detection based on machine vision. J Chin Image Graph 22(12):1640–1663. https://doi.org/10.11834/jig.160623
Ashour MW, Khalid F, Halin AA, Abdullah LN, Darwish SH (2018) Surface defects classification of hot-rolled steel strips using multi-directional shearlet features. Arabian Journal for Science & Engineering 44:2925–2932. https://doi.org/10.1007/s13369-018-3329-5
Luo Q, Sun Y, Li P, Simpson O, He Y (2018) Generalized completed local binary patterns for time-efficient steel surface defect classification. IEEE Trans Instrum Meas 99:1–13
Li M, Wan SH, Deng ZM, Wang YJ (2019) Fabric defect detection based on saliency histogram features. Comput Intell-Us 35(3):517–534. https://doi.org/10.1111/coin.12206
Luo Q, Fang X, Sun Y, Liu L, Simpson O (2019) Surface defect classification for hot-rolled steel strips by selectively dominant local binary patterns. IEEE Access 99:1–1
Li WC, Tsai DM (2012) Wavelet-based defect detection in solar wafer images with inhomogeneous texture. Pattern Recogn 45(2):742–756. https://doi.org/10.1016/j.patcog.2011.07.025
Malek AS, Drean JY, Bigue L, Osselin JF (2013) Optimization of automated online fabric inspection by fast Fourier transform (FFT) and cross-correlation. Text Res J 83(3):256–268. https://doi.org/10.1177/0040517512458340
Bissi L, Baruffa G, Placidi P, Ricci E, Scorzoni A, Valigi P (2013) Automated defect detection in uniform and structured fabrics using Gabor filters and PCA. J Vis Commun Image Represent 24(7):838–845
Hu GH, Zhang GH, Wang QH (2014) Automated defect detection in textured materials using wavelet-domain hidden Markov models. Opt Eng 53(9):093107
Wen ZJ, Cao JJ, Liu XP, Ying SH (2014) Fabric defects detection using adaptive wavelets. Int J Cloth Sci Technol 26(3):202–211. https://doi.org/10.1108/Ijcst-03-2013-0031
Hu GH, Wang QH, Zhang GH (2015) Unsupervised defect detection in textiles based on Fourier analysis and wavelet shrinkage. Appl Opt 54(10):2963–2980. https://doi.org/10.1364/Ao.54.002963
Bi X, Xu XP, Shen JH (2015) An automatic detection method of Mura defects for liquid crystal display using real Gabor filters. In: 2015 8th International Congress on Image and Signal Processing (Cisp), pp 871–875. https://doi.org/10.1109/CISP.2015.7408000
Hu GH (2015) Automated defect detection in textured surfaces using optimal elliptical Gabor filters. Optik 126(14):1331–1340. https://doi.org/10.1016/j.ijleo.2015.04.017
Tong L, Wong WK, Kwong CK (2016) Differential evolution-based optimal Gabor filter model for fabric inspection. Neurocomputing 173:1386–1401. https://doi.org/10.1016/j.neucom.2015.09.011
Chol DC, Jeon YJ, Kim SH, Moon S, Yun JP, Kim SW (2017) Detection of pinholes in steel slabs using Gabor filter combination and morphological features. ISIJ Int 57(6):1045–1053. https://doi.org/10.2355/isijinternational.ISIJINT-2016-160
Ma JX, Wang YX, Shi C, Lu CW (2018) Fast surface defect detection using improved Gabor filters. In: 2018 25th Ieee International Conference on Image Processing (Icip), pp 1508–1512. https://doi.org/10.1109/ICIP.2018.8451351
Ren RX, Hung T, Tan KC (2018) A generic deep-learning-based approach for automated surface inspection. IEEE Trans Cybern 48(3):929–940. https://doi.org/10.1109/Tcyb.2017.2668395
Kindermann R, Snell JL (1980) Markov random fields and their applications
Comer ML, Delp EJ (1999) Segmentation of textured images using a multiresolution Gaussian autoregressive model. IEEE Trans Image Process 8(3):408–420
Cohen FS, Fan Z, Attali S (1991) Automated inspection of textile fabrics using textural models. IEEE Trans Pattern Anal Mach Intell 13(8):803–808
Xu LJ, Huang Q (2012) Modeling the interactions among neighboring nanostructures for local feature characterization and defect detection. IEEE Trans Autom Sci Eng 9(4):745–754. https://doi.org/10.1109/Tase.2012.2209417
Kulkarni R, Banoth E, Pal P (2019) Automated surface feature detection using fringe projection: an autoregressive modeling-based approach. Opt Lasers Eng 121:506–511. https://doi.org/10.1016/j.optlaseng.2019.05.014
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175–185
Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca Raton
Jia HB, Murphey YL, Shi JJ, Chang TS (2004) An intelligent real-time vision system for surface defect detection. Int C Patt Recog:239–242. doi: https://doi.org/10.1109/Icpr.2004.1334512
Gao XD, Gao B, He Z, Xin WH (2006) Fabric defect detection based on support vector machine. J Text Res 27(5):26–28
Kang SB, Lee JH, Song KY, Pahk HJ (2009) Automatic defect classification of TFT-LCD panels using machine learning. In: 2009 IEEE International Symposium on Industrial Electronics, pp 2175–2177. https://doi.org/10.1109/ISIE.2009.5213760
Baly R, Hajj H (2012) Wafer classification using support vector machines. IEEE Trans Semicond Manuf 25(3):373–383. https://doi.org/10.1109/Tsm.2012.2196058
Huang W, Lu H (2013) Automatic defect classification of TFT-LCD panels with shape, histogram and color features. Int J Image Graph 13(03):1350011. https://doi.org/10.1142/S0219467813500113
Xie LJ, Huang R, Cao ZQ (2013) Detection and classification of defect patterns in optical inspection using support vector machines. Lect Notes Comput Sci 7995:376–384
Zhang ZQ, Wang XD, Liu S, Sun L, Sun LY, Guo YM (2018) An automatic recognition method for PCB visual defects. In: 2018 International Conference on Sensing, Diagnostics, Prognostics, and Control (Sdpc), pp 138–142. https://doi.org/10.1109/Sdpc.2018.00034
Kumar A (2003) Neural network based detection of local textile defects. Pattern Recogn 36(7):1645–1659. https://doi.org/10.1016/S0031-3203(03)00005-0
Kang GW, Liu HB (2005) Surface defects inspection of cold rolled strips based on neural network. In: 2005 International Conference on Machine Learning and Cybernetics 8:5034–5037. https://doi.org/10.1109/ICMLC.2005.1527830
Yang CH, Zhang JX, Ji G, Fu YJ, Hong X (2007) Recognition of defects in steel surface image based on neural networks and morphology. In: Second Workshop on Digital Media and Its Application in Museum & Heritage, Proceedings, pp 72–75. https://doi.org/10.1109/Dmamh.2007.56
Ashour MW, Hussin MF, Mahar KM (2008) Supervised texture classification using several features extraction techniques based on ANN and SVM. I C Comput Syst Appl:567–574. https://doi.org/10.1109/Aiccsa.2008.4493588
Chen LF, Su CT, Chen MH (2009) A neural-network approach for defect recognition in TFT-LCD photolithography process. IEEE Trans Electron Packag Manuf 32(1):1–8
Tseng DC, Chung IL, Tsai PL, Chou CM (2011) Defect classification for Lcd color filters using neural-network decision tree classifier. Int J Innov Comput I 7(7a):3695–3707
Kwon BG, Kang DJ (2011) Fast defect detection algorithm on the variety surface with random forest using GPUs. In: 2011 11th International Conference on Control, Automation and Systems (Iccas), pp 1135–1136
Tseng DC, Liu YS, Chou CM (2015) Automatic finger interruption detection in electroluminescence images of multicrystalline solar cells. Math Probl Eng 2015:1–12. https://doi.org/10.1155/2015/879675
Hu H, Liu Y, Liu M, Nie L (2016) Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm. Neurocomputing 181:86–95
Tian SY, Xu K (2017) An algorithm for surface defect identification of steel plates based on genetic algorithm and extreme learning machine. Metals-Basel 7(8). https://doi.org/10.3390/met7080311
Piao M, Jin CH, Lee JY, Byun JY (2018) Decision tree ensemble-based wafer map failure pattern recognition based on radon transform-based features. IEEE Trans Semicond Manuf 31(2):250–257. https://doi.org/10.1109/Tsm.2018.2806931
Celik HI, Dulger LU, Topalbekiroglu M (2014) Development of a machine vision system: real-time fabric defect detection and classification with neural networks. J Text Inst 105(6):575–585. https://doi.org/10.1080/00405000.2013.827393
Wang CH, Wang SJ, Lee WD (2006) Automatic identification of spatial defect patterns for semiconductor manufacturing. Int J Prod Res 44(23):5169–5185. https://doi.org/10.1080/02772240600610822
Nguyen VH, Pham VH, Cui X, Ma M, Kim H (2017) Design and evaluation of features and classifiers for OLED panel defect recognition in machine vision. J Inf Telecommun:334–350. https://doi.org/10.1080/24751839.2017.1355717
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal 35(8):1798–1828. https://doi.org/10.1109/Tpami.2013.50
Deng J, Dong W, Socher R, Li LJ, Li K, Li FF (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848
Feng X, Jiang Y, Yang X, Du M, Li X (2019) Computer vision algorithms and hardware implementations: a survey. Integration. 69:309–320. https://doi.org/10.1016/j.vlsi.2019.07.005
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: 2012 25th International Conference on Neural Information Processing Systems 1:1097–1105. https://doi.org/10.5555/2999134.2999257
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. https://arxiv.org/abs/1409.1556
Szegedy C, Liu W, Jia YQ, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr), pp 1–9. https://doi.org/10.1109/CVPR.2015.7298594
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr), pp 2818–2826. https://doi.org/10.1109/Cvpr.2016.308
He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. In: 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr), pp 770–778. https://doi.org/10.1109/Cvpr.2016.90
Huang G, Liu Z, van der Maaten L, Weinberger K (2017) Densely connected convolutional networks. In: Conference on Computer Vision and Pattern Recognition, 2017. https://doi.org/10.1109/CVPR.2017.243
Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size. arXiv:1602.07360. https://arxiv.org/abs/1602.07360
Howard AG, Zhu M, Bo C, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861. https://arxiv.org/abs/1704.04861
Sifre L (2014) Rigid-motion scattering for image classification. Ecole Polytechnique, Paris
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 4510–4520. https://doi.org/10.1109/CVPR.2018.00474
Zhang X, Zhou XY, Lin MX, Sun R (2018) ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr), pp 6848–6856. https://doi.org/10.1109/Cvpr.2018.00716
Ma N, Zhang X, Zheng H-T, Jian S (2018) ShuffleNet V2: practical guidelines for efficient CNN architecture design. In: 2018 European Conference on Computer Vision (ECCV). https://doi.org/10.1007/978-3-030-01264-9_8
Ren SQ, He KM, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Girshick R (2015) Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp 1440–1448. https://doi.org/10.1109/ICCV.2015.169
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr), pp 779–788. https://doi.org/10.1109/Cvpr.2016.91
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: Single Shot MultiBox Detector. Computer vision - Eccv 2016, Pt I 9905:21–37. https://doi.org/10.1007/978-3-319-46448-0_2
Rumelhart DE (1986) Learning representations by back-propagating errors. Nature. https://doi.org/10.1016/B978-1-4832-1446-7.50035-2
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507. https://doi.org/10.1126/science.1127647
Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: the 25th International Conference on Machine Learning (ICML 2008), pp 1096–1103. https://doi.org/10.1145/1390156.1390294
Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Proces Syst 27(nips 2014):27
Schlegl T, Seebock P, Waldstein SM, Langs G, Schmidt-Erfurth U (2019) f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med Image Anal 54:30–44. https://doi.org/10.1016/j.media.2019.01.010
Akcay S, Atapour-Abarghouei A, Breckon TP (2019) GANomaly: semi-supervised anomaly detection via adversarial training. Computer vision - Accv 2018, Pt Iii 11363:622–637. https://doi.org/10.1007/978-3-030-20893-6_39
DAGM texture dataset. https://hci.iwr.uni-heidelberg.de/node/3616. Accessed Oct. 2019
Wu MJ, Jang JSR, Chen JL (2015) Wafer map failure pattern recognition and similarity ranking for large-scale data sets. IEEE Trans Semicond Manuf 28(1):1–12. https://doi.org/10.1109/Tsm.2014.2364237
Song KC, Yan YH (2013) A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Appl Surf Sci 285:858–864. https://doi.org/10.1016/j.apsusc.2013.09.002
Tang S, He F, Huang X, Yang J (2019) Online PCB defect detector on a new PCB defect dataset
Huang YB, Qiu CY, Guo Y, Wang XN, Yuan K (2018) Surface defect saliency of magnetic tile. Ieee Int Con Auto Sc:612–617
Deitsch S, Christlein V, Berger S, Buerhop-Lutz C, Maier A, Gallwitz F, Riess C (2019) Automatic classification of defective photovoltaic module cells in electroluminescence images. Sol Energy 185:455–468
Gan JR, Li QT, Wang JZ, Yu HM (2017) A hierarchical extractor-based visual rail surface inspection system. IEEE Sensors J 17(23):7935–7944. https://doi.org/10.1109/Jsen.2017.2761858
TILDA Textile Texture-Database (1996). https://lmb.informatik.uni-freiburg.de/resources/datasets/tilda.en.html. Accessed Oct. 2019 2019
Kylberg G (2011) The Kylberg Texture Dataset v. 1.0. External report (Blue series) vol No. 35. Centre for Image Analysis, Swedish University of Agricultural Sciences and Uppsala University. http://www.cb.uu.se/~gustaf/texture/. Accessed 19 Jan 2021
Kampouris C, Zafeiriou S, Ghosh A, Malassiotis S (2016) Fine-grained material classification using micro-geometry and reflectance. Computer vision - Eccv 2016, Pt V 9909:778–792. https://doi.org/10.1007/978-3-319-46454-1_47
Fritz M, Hayman E, Caputo B, Eklundh J-O (2019) The KTH-TIPS database. Accessed Oct. 2019
Li YY, Zhang D, Lee DJ (2019) Automatic fabric defect detection with a wide-and-compact network. Neurocomputing 329:329–338. https://doi.org/10.1016/j.neucom.2018.10.070
Michalski P, Ruszczak B, Tomaszewski M (2018) Convolutional neural networks implementations for computer vision. Adv Intell Syst 720:98–110. https://doi.org/10.1007/978-3-319-75025-5_10
Caggiano A, Zhang JJ, Alfieri V, Caiazzo F, Gao R, Teti R (2019) Machine learning-based image processing for on-line defect recognition in additive manufacturing. Cirp Ann Manuf Technol 68(1):451–454. https://doi.org/10.1016/j.cirp.2019.03.021
Yang H, Mei S, Song K, Tao B, Yin Z (2018) Transfer-learning-based online Mura defect classification. IEEE Trans Semicond Manuf 31(1):116–123. https://doi.org/10.1109/TSM.2017.2777499
Kim Y-G, Lim D-U, Ryu J-H, Park T-H SMD Defect classification by convolution neural network and PCB image transform. In: 3rd IEEE International Conference on Computing, Communication and Security, ICCCS 2018, October 25, 2018 - October 27, 2018, Kathmandu, Nepal, 2018. Proceedings on 2018 IEEE 3rd International Conference on Computing, Communication and Security, ICCCS 2018. Institute of Electrical and Electronics Engineers Inc, pp 180–183. https://doi.org/10.1109/CCCS.2018.8586818
Kim J, Kim S, Kwon N, Kang H, Kim Y, Lee C Deep learning based automatic defect classification in through-silicon Via process: FA: Factory automation. In: 29th Annual SEMI Advanced Semiconductor Manufacturing Conference, Saratoga Springs, NY, United states, 2018 2018. 2018 29th Annual SEMI Advanced Semiconductor Manufacturing Conference, ASMC 2018. Institute of Electrical and Electronics Engineers Inc, pp 35–39. https://doi.org/10.1109/ASMC.2018.8373144
Jang C, Yun S, Hwang H, Shin H, Kim SS, Park Y (2018) A defect inspection method for machine vision using defect probability image with deep convolutional neural network. In: 2018 Asian Conference on Computer Vision (ACCV ), pp 142–154. https://doi.org/10.1007/978-3-030-20887-5_9
Zhang L, Jin Y, Yang X, Li X, Duan X, Sun Y, Liu H (2018) Convolutional neural network-based multi-label classification of PCB defects. J Eng 16:1612–1616. https://doi.org/10.1049/joe.2018.8279
Deng Y-S, Luo A-C, Dai M-J Building an automatic defect verification system using deep neural network for PCB defect classification. In: 4th International Conference on Frontiers of Signal Processing, ICFSP 2018, September 24, 2018 - September 27, 2018, Poitiers, France, 2018. 2018 4th International Conference on Frontiers of Signal Processing, ICFSP 2018. Institute of Electrical and Electronics Engineers Inc, pp 145–149. https://doi.org/10.1109/ICFSP.2018.8552045
Ghosh B, Bhuyan MK, Sasmal P, Iwahori Y, Gadde P Defect classification of printed circuit boards based on transfer learning. In: 2018 IEEE Applied Signal Processing Conference, ASPCON 2018, December 7, 2018 - December 9, 2018, Kolkata, India, 2018. Proceedings of 2018 IEEE Applied Signal Processing Conference, ASPCON 2018. Institute of Electrical and Electronics Engineers Inc, pp 245–248. https://doi.org/10.1109/ASPCON.2018.8748670
Wei P, Liu C, Liu M, Gao Y, Liu H (2018) CNN based reference comparison method for classifying bare PCB defects. J Eng 2018(16):1528–1533. https://doi.org/10.1049/joe.2018.8271
Nakazawa T, Kulkarni DV (2018) Wafer map defect pattern classification and image retrieval using convolutional neural network. IEEE Trans Semicond Manuf 31(2):309–314. https://doi.org/10.1109/TSM.2018.2795466
Yuan-Fu Y (2019) A deep learning model for identification of defect patterns in semiconductor wafer map. In: 30th Annual SEMI Advanced Semiconductor Manufacturing Conference, ASMC 2019. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ASMC.2019.8791815
Ishida T, Nitta I, Fukuda D, Kanazawa Y (2019) Deep learning-based wafer-map failure pattern recognition framework. In: 2019 20th International Symposium on Quality Electronic Design (Isqed), pp 291–297. https://doi.org/10.1109/ISQED.2019.8697407
Cheon S, Lee H, Kim CO, Lee SH (2019) Convolutional neural network for wafer surface defect classification and the detection of unknown defect class. IEEE Trans Semicond Manuf 32(2):163–170. https://doi.org/10.1109/Tsm.2019.2902657
Banda P, Barnard L A deep learning approach to photovoltaic cell defect classification. In: 2018 Annual Conference of the South African Institute of Computer Scientists and Information Technologists: Technology for Change, Port Elizabeth, South Africa, 2018 2018. ACM International Conference Proceeding Series. Association for Computing Machinery, pp 215–221. https://doi.org/10.1145/3278681.3278707
Lin H, Li B, Wang XG, Shu YF, Niu SL (2019) Automated defect inspection of LED chip using deep convolutional neural network. J Intell Manuf 30(6):2525–2534. https://doi.org/10.1007/s10845-018-1415-x
Park JK, Kwon BK, Park JH, Kang DJ (2016) Machine learning-based imaging system for surface defect inspection. Int J Precis Eng Manuf Green Technol 3(3):303–310
Weimer D, Scholz-Reiter B, Shpitalni M (2016) Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. Cirp Ann Manuf Technol 65(1):417–420. https://doi.org/10.1016/j.cirp.2016.04.072
Wang T, Chen Y, Qiao MN, Snoussi H (2018) A fast and robust convolutional neural network-based defect detection model in product quality control. Int J Adv Manuf Technol 94(9–12):3465–3471. https://doi.org/10.1007/s00170-017-0882-0
Jeyaraj PR, Samuel Nadar ER (2019) Computer vision for automatic detection and classification of fabric defect employing deep learning algorithm. Int J Cloth Sci Technol 31(4):510–521. https://doi.org/10.1108/IJCST-11-2018-0135
Saiz FA, Serrano I, Barandiaran I, Sanchez JR A robust and fast deep learning-based method for defect classification in steel surfaces. In: 9th International Conference on Intelligent Systems, IS 2018, September 25, 2018 - September 27, 2018, Funchal - Madeira, Portugal, 2018. 9th International Conference on Intelligent Systems 2018: Theory, Research and Innovation in Applications, IS 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc, pp 455–460. https://doi.org/10.1109/IS.2018.8710501
Chen W, Gao Y, Gao L, Li XA (2018) New ensemble approach based on deep convolutional neural networks for steel surface defect classification. In: 51st CIRP Conference on Manufacturing Systems, CIRP CMS 2018, May 16, 2018 - May 18, 2018, Stockholm, Sweden. Elsevier B.V, pp 1069–1072. https://doi.org/10.1016/j.procir.2018.03.264
Liu Z, Wang X, Chen X Inception dual network for steel strip defect detection. In: 16th IEEE International Conference on Networking, Sensing and Control, ICNSC 2019, May 9, 2019 - May 11, 2019, Banff, AB, Canada, 2019. Proceedings of the 2019 IEEE 16th International Conference on Networking, Sensing and Control, ICNSC 2019. Institute of Electrical and Electronics Engineers Inc, pp 409–414. https://doi.org/10.1109/ICNSC.2019.8743190
Vannocci M, Ritacco A, Castellano A, Galli F, Vannucci M, Iannino V, Colla V Flatness defect detection and classification in hot rolled steel strips using convolutional neural networks. In: 15th International Work-Conference on Artificial Neural Networks, IWANN 2019, June 12, 2019 - June 14, 2019, Gran Canaria, Spain, 2019. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp 220–234. https://doi.org/10.1007/978-3-030-20518-8_19
Song LM, Li XY, Yang YG, Zhu XJ, Guo QH, Yang HD (2018) Detection of micro-defects on metal screw surfaces based on deep convolutional neural networks. Sensors-Basel 18(11). https://doi.org/10.3390/s18113709
Chun LP, Zhao QF (2018) Product surface defect detection based on deep learning. In: 2018 16th Ieee Int Conf on Dependable, Autonom and Secure Comp, pp 250–255. https://doi.org/10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.00051
Soukup D, Huber-Mork R (2014) Convolutional neural networks for steel surface defect detection from photometric stereo images. Advances in visual computing (Isvc 2014), Pt 1 8887:668–677
Mei S, Wang YD, Wen GJ (2018) Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model. Sensors-Basel 18(4). https://doi.org/10.3390/s18041064
Mujeeb A, Dai WT, Erdt M, Sourin A (2018) Unsupervised surface defect detection using deep autoencoders and data augmentation. In: 2018 International Conference on Cyberworlds (Cw), pp 391–398. https://doi.org/10.1109/Cw.2018.00076
Siegmund D, Prajapati A, Kirchbuchner F, Kuijper A (2018) An integrated deep neural network for defect detection in dynamic textile textures. In: Progress in Artificial Intelligence and Pattern Recognition, Iwaipr 2018, vol 11047, pp 77–84. https://doi.org/10.1007/978-3-030-01132-1_9
Li JY, Su ZF, Geng JH, Yin YX (2018) Real-time detection of steel strip surface defects based on improved YOLO detection network. IFAC-PapersOnLine 51(21):76–81. https://doi.org/10.1016/j.ifacol.2018.09.412
Li YT, Huang HS, Xie QS, Yao LG, Chen QP (2018) Research on a surface defect detection algorithm based on MobileNet-SSD. Appl Sci-Basel 8(9). https://doi.org/10.3390/app8091678
Yang J, Li S, Wang Z, Yang G (2019) Real-time tiny part defect detection system in manufacturing using deep learning. IEEE Access 7:89278–89291. https://doi.org/10.1109/ACCESS.2019.2925561
Di H, Ke X, Peng Z, Zhou D (2019) Surface defect classification of steels with a new semi-supervised learning method. Opt Lasers Eng 117(1):40–48
Gao YP, Gao L, Li XY, Yan XG (2020) A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robot Comput Integr Manuf 61:101825. https://doi.org/10.1016/j.rcim.2019.101825
Tan CQ, Sun FC, Kong T, Zhang WC, Yang C, Liu CF (2018) A survey on deep transfer learning. Artificial neural networks and machine learning - Icann 2018, Pt Iii 11141:270–279. https://doi.org/10.1007/978-3-030-01424-7_27
Liu SP, Tian GH, Xu Y (2019) A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter. Neurocomputing 338:191–206. https://doi.org/10.1016/j.neucom.2019.01.090
Zheng X, Chen J, Wang H, Zheng S, Kong Y (2020) A deep learning-based approach for the automated surface inspection of copper clad laminate images. Applied Intelligence. https://doi.org/10.1007/s10489-020-01877-z
Zhu XJ (2005) Semi-supervised learning literature survey. University of Wisconsin-Madison Department of Computer Sciences. http://digital.library.wisc.edu/1793/60444. Accessed 19 Jan 2021
Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning, vol 2. MIT Press Cortes, Cambridge
Cortes C, Mohri M (2014) Domain adaptation and sample bias correction theory and algorithm for regression. Theor Comput Sci 519:103126
Odena A (2016) Semi-supervised learning with generative adversarial networks. arXiv:1606.01583. https://arxiv.org/abs/1606.01583
Li W, Wang Z, Li J, Polson J, Speier W, Arnold CW (2019) Semi-supervised learning based on generative adversarial network: a comparison between good GAN and bad GAN approach. In: 2019 Computer Vision and Pattern Recognition (CVPR) Workshops. arXiv:1905.06484. https://arxiv.org/abs/1905.06484
Berthelot D, Carlini N, Goodfellow I, Papernot N, Oliver A, Raffel CA (2019) Mixmatch: a holistic approach to semi-supervised learning. In: Advances in Neural Information Processing Systems, pp 5049–5059
Zheng X, Wang H, Chen J, Kong Y, Zheng S (2020) A generic semi-supervised deep learning-based approach for automated surface inspection. IEEE Access 8:114088–114099
Funding
This work was supported in part by the National Natural Science Foundation of China under grant number U1609212, Zhejiang Provincial Science and Technology Plan under grant number 2019C04021, and Zhejiang Province Public Technology Research Project under grant number LGG20F030002.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
Not applicable.
Consent to participate
Not applicable.
Consent to publish
Not applicable.
Code availability
Not applicable.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zheng, X., Zheng, S., Kong, Y. et al. Recent advances in surface defect inspection of industrial products using deep learning techniques. Int J Adv Manuf Technol 113, 35–58 (2021). https://doi.org/10.1007/s00170-021-06592-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00170-021-06592-8