{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T16:53:39Z","timestamp":1726851219634},"reference-count":36,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T00:00:00Z","timestamp":1649289600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"This paper addresses the problem of automatic quality inspection in assembly processes by discussing the design of a computer vision system realized by means of a heterogeneous multiprocessor system-on-chip. Such an approach was applied to a real catalytic converter assembly process, to detect planar, translational, and rotational shifts of the flanges welded on the central body. The manufacturing line imposed tight time and room constraints. The image processing method and the features extraction algorithm, based on a specific geometrical model, are described and validated. The algorithm was developed to be highly modular, thus suitable to be implemented by adopting a hardware\u2013software co-design strategy. The most timing consuming computational steps were identified and then implemented by dedicated hardware accelerators. The entire system was implemented on a Xilinx Zynq heterogeneous system-on-chip by using a hardware\u2013software (HW\u2013SW) co-design approach. The system is able to detect planar and rotational shifts of welded flanges, with respect to the ideal positions, with a maximum error lower than one millimeter and one sexagesimal degree, respectively. Remarkably, the proposed HW\u2013SW approach achieves a 23\u00d7 speed-up compared to the pure software solution running on the Zynq embedded processing system. Therefore, it allows an in-line automatic quality inspection to be performed without affecting the production time of the existing manufacturing process.<\/jats:p>","DOI":"10.3390\/s22082839","type":"journal-article","created":{"date-parts":[[2022,4,8]],"date-time":"2022-04-08T01:08:22Z","timestamp":1649380102000},"page":"2839","source":"Crossref","is-referenced-by-count":8,"title":["Robust and High-Performance Machine Vision System for Automatic Quality Inspection in Assembly Processes"],"prefix":"10.3390","volume":"22","author":[{"given":"Fabio","family":"Frustaci","sequence":"first","affiliation":[{"name":"Department of Informatics, Modeling, Electronics and Systems Engineering, University of Calabria, 87036 Rende, Italy"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-2197-4563","authenticated-orcid":false,"given":"Fanny","family":"Spagnolo","sequence":"additional","affiliation":[{"name":"Department of Informatics, Modeling, Electronics and Systems Engineering, University of Calabria, 87036 Rende, Italy"}]},{"given":"Stefania","family":"Perri","sequence":"additional","affiliation":[{"name":"Department of Mechanical, Energy and Management Engineering, University of Calabria, 87036 Rende, Italy"}]},{"given":"Giuseppe","family":"Cocorullo","sequence":"additional","affiliation":[{"name":"Department of Informatics, Modeling, Electronics and Systems Engineering, University of Calabria, 87036 Rende, Italy"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9528-1110","authenticated-orcid":false,"given":"Pasquale","family":"Corsonello","sequence":"additional","affiliation":[{"name":"Department of Informatics, Modeling, Electronics and Systems Engineering, University of Calabria, 87036 Rende, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Tambare, P., Meshram, C., Lee, C.-C., Ramteke, R.J., and Imoize, A.L. (2022). Performance Measurement System and Quality Management in Data-Driven Industry 4.0: A Review. Sensors, 22.","DOI":"10.3390\/s22010224"},{"key":"ref_2","first-page":"631","article-title":"Machine Vision Systems for Industrial Quality Control Inspections","volume":"Volume 540","author":"Chiabert","year":"2018","journal-title":"Product Lifecycle Management to Support Industry 4.0, Proceedings of the IFIP Advances in Information and Communication Technology, Turin, Italy, 2\u20134 July 2018"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Golnabia, H., and Asadpourb, A. (2007). Design and application of industrial machine vision systems. Robotics and Computer-Integrated Manufacturing, Springer.","DOI":"10.1016\/j.rcim.2007.02.005"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Garc\u00eda-Alcaraz, J., Maldonado-Mac\u00edas, A., and Cortes-Robles, G. (2014). Automatic Product Quality Inspection Using Computer Vision Systems. Lean Manufacturing in the Developing World, Springer.","DOI":"10.1007\/978-3-319-04951-9"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chen, C., Zhang, C., Wang, T., Li, D., Guo, Y., Zhao, Z., and Hong, J. (2020). Monitoring of Assembly Process Using Deep Learning Technology. Sensors, 20.","DOI":"10.3390\/s20154208"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1016\/j.procir.2016.11.152","article-title":"The Digital Twin: Realizing the Cyber-Physical Production System for Industry 4.0","volume":"61","author":"Uhlemanna","year":"2017","journal-title":"Procedia CIRP"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.jmsy.2021.05.011","article-title":"Digital twins-based smart manufacturing system design in Industry 4.0: A review","volume":"60","author":"Leng","year":"2021","journal-title":"J. Manuf. Syst."},{"key":"ref_8","first-page":"828","article-title":"Computer vision algorithm for measurement and inspection of O-rings","volume":"94","author":"Gaoliang","year":"2016","journal-title":"Measurements"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/1687814017717183","article-title":"Development of a simple three-dimensional machine-vision measurement system for in-process mechanical parts","volume":"9","author":"Ngo","year":"2017","journal-title":"Adv. Mech. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Petri\u0161i\u010d, J., Suhadolnik, A., and Kosel, F. (2008, January 26\u201330). Object Length and Area Calculations on the Digital Images. Proceedings of the 12th International Conference Trends in the Development of Machinery and Associated Technology (TMT 2008), Istanbul, Turkey.","DOI":"10.1016\/j.imavis.2007.11.001"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.optmat.2014.11.020","article-title":"Effect of matte coating on 3D optical measurement accuracy","volume":"40","author":"Palousek","year":"2015","journal-title":"Opt. Mater."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1006\/rtim.2001.0256","article-title":"A Hybrid Image Alignment System for Fast and Precise Pattern Localization","volume":"8","author":"Lai","year":"2002","journal-title":"Real-Time Imaging"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1177\/0020294015602499","article-title":"A Machine Vision System for Tool Positioning and Its Verification","volume":"48","author":"Mahapatra","year":"2015","journal-title":"Meas. Control."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"8214","DOI":"10.1109\/TIE.2018.2881948","article-title":"Quality Inspection of Remote Radio Units Using Depth-Free Image-Based Visual Servo with Acceleration Command","volume":"66","author":"Anwar","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"9641","DOI":"10.1109\/TIE.2019.2896165","article-title":"A Smart Monitoring System for Automatic Welding Defect Detection","volume":"66","author":"Sassi","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.ndteint.2016.11.003","article-title":"Automated detection of welding defects in pipelines from radiographic images DWDI","volume":"86","author":"Boaretto","year":"2017","journal-title":"NDT&E Int."},{"key":"ref_17","first-page":"4498","article-title":"Inspection of Imprint Defects in Stamped Metal Surfaces Using Deep Learning and Tracking","volume":"66","author":"Block","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/j.procir.2014.10.025","article-title":"Dynamic Alignment Control Using Depth Imagery for Automated Wheel Assembly","volume":"25","author":"Prabhu","year":"2014","journal-title":"Procedia CIRP"},{"key":"ref_19","unstructured":"Fern\u00e1ndez, A., Acevedo, R.G., Alvarez, E.A., Lopez, A.C., Garcia, D.F., Fernandez, R.U., Meana, M.J., and Sanchez, J.M.G. (2009, January 4\u20138). Low-Cost System for Weld Tracking Based on Artificial Vision. Proceedings of the 2009 IEEE Industry Applications Society Annual Meeting, Houston, TX, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1016\/j.promfg.2015.09.051","article-title":"A Comparative Study of Machine Vision Based Methods for Fault Detection in an Automated Assembly Machine","volume":"1","author":"Chauhan","year":"2015","journal-title":"Procedia Manuf."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1954","DOI":"10.1109\/TIA.2013.2259786","article-title":"Jam Detector for Steel Pickling Lines Using Machine Vision","volume":"49","author":"Usamentiaga","year":"2013","journal-title":"IEEE Trans. Ind. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/j.promfg.2020.02.072","article-title":"An embedded machine vision system for an in-line quality check of assembly processes","volume":"42","author":"Frustaci","year":"2020","journal-title":"Procedia Manuf."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A Computational Approach to Edge Detection","volume":"8","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"156846","DOI":"10.1109\/ACCESS.2021.3130132","article-title":"An Adaptive Threshold for the Canny Algorithm with Deep Reinforcement Learning","volume":"9","author":"Choi","year":"2021","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"39934","DOI":"10.1109\/ACCESS.2020.2976860","article-title":"Noise-Robust, Reconfigurable Canny Edge Detection and its Hardware Realization","volume":"8","author":"Kalbasi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1109\/TRO.2018.2875382","article-title":"Canny-VO: Visual Odometry with RGB-D Cameras Based on Geometric 3-D\u20132-D Edge Alignment","volume":"35","author":"Zhou","year":"2019","journal-title":"IEEE Trans. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1037","DOI":"10.1109\/TCSVT.2016.2640038","article-title":"Energy Efficient Canny Edge Detector for Advanced Mobile Vision Applications","volume":"28","author":"Lee","year":"2018","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press. [2nd ed.].","DOI":"10.1017\/CBO9780511811685"},{"key":"ref_29","unstructured":"(2022, February 26). AMBA 4 AXI4, AXI4-Lite, and AXI4-Stream Protocol Assertions User Guide. Available online: http:\/\/infocenter.arm.com\/help\/index.jsp?topic=\/com.arm.doc.ihi0022d\/index.html."},{"key":"ref_30","unstructured":"(2022, February 26). Vivado Design Suite User Guide-High-Level Synthesis, UG902 (v2020.1), May 2021. Available online: https:\/\/www.xilinx.com\/support\/documentation\/sw_manuals\/xilinx2020_2\/ug902-vivado-high-level-synthesis.pdf."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Oh, S., You, J.-H., and Kim, Y.-K. (2019, January 15\u201318). FPGA Acceleration of Bolt Inspection Algorithm for a High-Speed Embedded Machine Vision System. Proceedings of the 2019 19th International Conference on Control, Automation and Systems (ICCAS 2019), Jeju, Korea.","DOI":"10.23919\/ICCAS47443.2019.8971760"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2117","DOI":"10.1109\/TIE.2021.3057026","article-title":"A novel defect detection algorithm for flexible integrated circuit package substrates","volume":"69","author":"Zhong","year":"2022","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Le, M.-T., Tu, C.-T., Guo, S.-M., and Lien, J.-J.J. (2020). A PCB Alignment System Using RST Template Matching with CUDA on Embedded GPU Board. Sensors, 20.","DOI":"10.3390\/s20092736"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4715","DOI":"10.1109\/JSEN.2018.2824660","article-title":"Feature Extraction of Welding Seam Image Based on Laser Vision","volume":"18","author":"Gu","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1366","DOI":"10.1109\/TIE.2019.2899555","article-title":"Design of a New Vision-Based Method for the Bolts Looseness Detection in Flange Connections","volume":"67","author":"Wang","year":"2020","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.promfg.2018.07.004","article-title":"Machine vision assisted micro-filament detection for real-time monitoring of electrohydrodynamic inkjet printing","volume":"26","author":"Lies","year":"2018","journal-title":"Procedia Manuf."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/8\/2839\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,28]],"date-time":"2024-07-28T07:01:28Z","timestamp":1722150088000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/8\/2839"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,7]]},"references-count":36,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["s22082839"],"URL":"https:\/\/doi.org\/10.3390\/s22082839","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,7]]}}}