Abstract
Nondestructive testing using X-ray imaging has been widely adopted in the defect detection of casting parts for quality management. Deep learning has been proved to be an effective way to detect defects in X-ray images. In this work, Feature Pyramid Network (FPN) which has been utilized broadly in many applications is adopted as our baseline. In FPN, there mainly exits two issues: firstly, down sampling operation in Convolutional Neural Network is often utilized to enhance the perception field, causing the loss of location information in feature maps, and secondly, there exists feature imbalance in feature maps and proposals. DetNet and Path Aggregation Network are adopted to solve the two shortages. To further improve the recall rate, soft Non-Maximum Suppression (soft-NMS) is adopted to remain more proposals that have high classification confidence. Defects in X-ray images of casting parts are provided with low semantic information, causing the different instances between detection results and annotations in the same area. We propose soft Intersection Over Union (soft-IOU) criterion which could evaluate several results or ground truths in the near area, making it more accurate to evaluate detection results. The experimental results demonstrate that the three proposed strategies have better performance than the baseline for our dataset.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Badmos, O., Kopp, A., Bernthaler, T., & Schneider, G. (2019). Image-based defect detection in lithium-ion battery electrode using convolutional neural networks. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-019-01476-x.
Bengio, Y. (2009). Learning deep architectures for AI. Foundations and Trends® in Machine Learning, 2(1), 1–127. https://doi.org/10.1561/2200000006.
Bodla, N., Singh, B., Chellappa, R., & Davis, L. S. (2017). Soft-NMS–improving object detection with one line of code. In Proceedings of the IEEE international conference on computer vision, (pp. 5561–5569). http://doi.org/10.1109/iccv.2017.593.
Cheng, J. C. P., & Wang, M. (2018). Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Automation in Construction, 95, 155–171. https://doi.org/10.1016/j.autcon.2018.08.006.
Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In IEEE computer society conference on computer vision and pattern recognition, CVPR 2005, (Vol. 1, pp. 886–893): IEEE. http://doi.org/10.1109/cvpr.2005.177.
D’Orazio, T., Leo, M., Distante, A., Guaragnella, C., Pianese, V., & Cavaccini, G. (2008). Automatic ultrasonic inspection for internal defect detection in composite materials. NDT & E International, 41(2), 145–154. https://doi.org/10.1016/j.ndteint.2007.08.001.
Du, W., Shen, H., Fu, J., Zhang, G., & He, Q. (2019). Approaches for improvement of the X-ray image defect detection of automobile casting aluminum parts based on deep learning. NDT & E International, 107, 102144. https://doi.org/10.1016/j.ndteint.2019.102144.
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2009). Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1627–1645. https://doi.org/10.1109/tpami.2009.167.
Ferguson, M., Ak, R., Lee, Y. T., & Law, K. H. (2017). Automatic localization of casting defects with convolutional neural networks. In 2017 IEEE international conference on big data (big data), 11–14 December 2017, (pp. 1726–1735). http://doi.org/10.1109/BigData.2017.8258115.
Ghorai, S., Mukherjee, A., Gangadaran, M., & Dutta, P. K. (2013). Automatic defect detection on hot-rolled flat steel products. IEEE Transactions on Instrumentation and Measurement, 62(3), 612–621. https://doi.org/10.1109/tim.2012.2218677.
Girshick, R. (2015). Fast R-CNN. In 2015 IEEE international conference on computer vision. IEEE, New York, pp. 1440–1448. http://dx.doi.org/10.1109/ICCV.2015.169.
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 580–587). http://doi.org/10.1109/cvpr.2014.81.
Grunwald, E., Hammer, R., Rosc, J., Maier, G. A., Barnthaler, M., Cordill, M. J., et al. (2016). Advanced 3D failure characterization in multi-layered PCBs. NDT & E International, 84, 99–107. https://doi.org/10.1016/j.ndteint.2016.08.003.
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP(99), 1. https://doi.org/10.1109/TPAMI.2018.2844175.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 770–778). http://doi.org/10.1109/CVPR.2016.90.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, (pp. 1097–1105) http://doi.org/10.1145/3065386.
Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., & Sun, J. (2018). Detnet: Design backbone for object detection. In Proceedings of the European conference on computer vision (ECCV), (pp. 334–350).
Lin, T.-Y., Dollár, P., Girshick, R. B., He, K., Hariharan, B., & Belongie, S. J. (2017). Feature pyramid networks for object detection. In CVPR, (Vol. 1, p. 4, Vol. 2) http://doi.org/10.1109/CVPR.2017.106.
Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2018). Automated defect inspection of LED chip using deep convolutional neural network. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1415-x.
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., et al. (2016). SSD: Single Shot MultiBox Detector. In European Conference on Computer Vision, (pp. 21–37) http://dx.doi.org/10.1007/978-3-319-46448-0_2.
Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018). Path aggregation network for instance segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 8759–8768) http://doi.org/10.1109/cvpr.2018.00913.
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110. https://doi.org/10.1023/b:visi.0000029664.99615.94.
Luo, H., Gu, Y., Liao, X., Lai, S., & Jiang, W. (2019). Bag of tricks and a strong baseline for deep person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, (p. 0).
Maire, E., & Withers, P. J. (2014). Quantitative X-ray tomography. International Materials Reviews, 59(1), 1–43. https://doi.org/10.1179/1743280413y.0000000023.
Mery, D. (2014). Computer vision technology for X-ray testing. Insight, 56(3), 147–154. https://doi.org/10.1784/insi.2014.56.3.147.
Mery, D., & Arteta, C. (2017) Automatic defect recognition in x-ray testing using computer vision. In 2017 IEEE winter conference on applications of computer vision (WACV), (pp. 1026–1035), IEEE http://doi.org/10.1109/wacv.2017.119.
Mery, D., Riffo, V., Zscherpel, U., Mondragon, G., Lillo, I., Zuccar, I., et al. (2015). GDXray: The Database of X-ray Images for Nondestructive Testing. Journal of Nondestructive Evaluation, 34(4), 12. https://doi.org/10.1007/s10921-015-0315-7.
Ojala, T., Pietikainen, M., & Harwood, D. (1996). A comparative study of texture measures with classification based on feature distributions. Pattern Recognition, 29(1), 51–59. https://doi.org/10.1016/0031-3203(95)00067-4.
Palumbo, D., Tamborrino, R., Galietti, U., Aversa, P., Tati, A., & Luprano, V. A. M. (2016). Ultrasonic analysis and lock-in thermography for debonding evaluation of composite adhesive joints. NDT & E International, 78, 1–9. https://doi.org/10.1016/j.ndteint.2015.09.001.
Rajkolhe, R., & Khan, J. (2014). Defects, causes and their remedies in casting process: A review. International Journal of Research in Advent Technology, 2(3), 375–383.
Ren, S. Q., He, K. M., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/tpami.2016.2577031.
Shanmugamani, R., Sadique, M., & Ramamoorthy, B. (2015). Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement, 60, 222–230. https://doi.org/10.1016/j.measurement.2014.10.009.
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
Staude, A., Bartscher, M., Ehrig, K., Goebbels, J., Koch, M., Neuschaefer-Rube, U., et al. (2011). Quantification of the capability of micro-CT to detect defects in castings using a new test piece and a voxel-based comparison method. NDT & E International, 44(6), 531–536. https://doi.org/10.1016/j.ndteint.2011.05.006.
Wei, X., Yang, Z., Liu, Y., Wei, D., Jia, L., & Li, Y. (2019). Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study. Engineering Applications of Artificial Intelligence, 80, 66–81.
Xia, N., Zhao, P., Xie, J., Zhang, C., Fu, J., & Turng, L.-S. (2018). Defect diagnosis for polymeric samples via magnetic levitation. NDT & E International, 100, 175–182. https://doi.org/10.1016/j.ndteint.2018.09.004.
Xu, X. J., Lei, Y., & Yang, F. (2018). Railway subgrade defect automatic recognition method based on improved faster R-CNN. Scientific Programming. https://doi.org/10.1155/2018/4832972.
Yu, F., & Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122.
Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. In European conference on computer vision, (pp. 818–833): Springer. http://doi.org/10.1007/978-3-319-10590-1_53.
Zhu, L., Chen, Y., Yuille, A., & Freeman, W. (2010) Latent hierarchical structural learning for object detection. In 2010 IEEE computer society conference on computer vision and pattern recognition, (pp. 1062–1069): IEEE http://doi.org/10.1109/cvpr.2010.5540096.
Acknowledgments
This work was financially supported by the National Nature Science Foundation of China (No. 51975518), the Science Fund for Creative Research Groups of National Natural Science Foundation of China (No. 51821093), Key Research and Development Plan of Zhejiang Province (No 2018C01073), Ningbo Science and Technology Plan Project (2019B10072), and the Fundamental Research Funds for the Central Universities (No. 2019QNA4004).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Du, W., Shen, H., Fu, J. et al. Automated detection of defects with low semantic information in X-ray images based on deep learning. J Intell Manuf 32, 141–156 (2021). https://doi.org/10.1007/s10845-020-01566-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-020-01566-1