{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,4,4]],"date-time":"2025-04-04T21:04:34Z","timestamp":1743800674580,"version":"3.37.3"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T00:00:00Z","timestamp":1665964800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T00:00:00Z","timestamp":1665964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010061","name":"University of Waikato","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100010061","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Intell Robot Appl"],"published-print":{"date-parts":[[2023,6]]},"abstract":"Abstract<\/jats:title>Stereo vision system and manipulator are two major components of an autonomous fruit harvesting system. In order to raise the fruit-harvesting rate, stereo vision system calibration and kinematic calibration are two significant processes to improve the positional accuracy of the system. This article reviews the mathematics of these two calibration processes and presents an integrated approach for acquiring calibration data and calibrating both components of an autonomous kiwifruit harvesting system. The calibrated harvesting system yields good positional accuracy in the laboratory tests, especially in harvesting individual kiwifruit. However, the performance is not in line with the outcomes in the orchard field tests due to the cluster growing style of kiwifruit. In the orchard test, the calibrations reduce the fruit drop rate but it does not impressively raise the fruit harvesting rate. Most of the fruit in the clusters remain in the canopy due to the invisibility of the stereo vision system. After analyzing the existing stereo vision system, a\u00a0future visual sensing system research direction for an autonomous fruit harvesting system is justified.<\/jats:p>","DOI":"10.1007\/s41315-022-00263-x","type":"journal-article","created":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T07:02:36Z","timestamp":1665990156000},"page":"350-369","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Integration of stereo vision system calibration and kinematic calibration for an autonomous kiwifruit harvesting system"],"prefix":"10.1007","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5509-3684","authenticated-orcid":false,"given":"Chi Kit","family":"Au","sequence":"first","affiliation":[]},{"given":"Shen Hin","family":"Lim","sequence":"additional","affiliation":[]},{"given":"Mike","family":"Duke","sequence":"additional","affiliation":[]},{"given":"Ye Chow","family":"Kuang","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Redstall","sequence":"additional","affiliation":[]},{"given":"Canaan","family":"Ting","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,10,17]]},"reference":[{"issue":"8","key":"263_CR1","first-page":"855","volume":"37","author":"DC Brown","year":"1971","unstructured":"Brown, D.C.: Close-range camera calibration. Photogramm. Eng. 37(8), 855\u2013866 (1971)","journal-title":"Photogramm. Eng."},{"issue":"4","key":"263_CR2","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MRA.2006.250573","volume":"13","author":"F Chaumette","year":"2006","unstructured":"Chaumette, F., Hutchinson, S.: Visual servo control. I. Basic approaches. IEEE Robot. Autom. Mag. 13(4), 82\u201390 (2006). https:\/\/doi.org\/10.1109\/MRA.2006.250573","journal-title":"IEEE Robot. Autom. Mag."},{"issue":"1","key":"263_CR3","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1109\/MRA.2007.339609","volume":"14","author":"F Chaumette","year":"2007","unstructured":"Chaumette, F., Hutchinson, S.: Visual servo control. II. Advanced approaches. IEEE Robot. Autom. Mag. 14(1), 109\u2013118 (2007). https:\/\/doi.org\/10.1109\/MRA.2007.339609","journal-title":"IEEE Robot. Autom. Mag."},{"issue":"9","key":"263_CR4","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1080\/09500340.2015.1106602","volume":"63","author":"J Cui","year":"2016","unstructured":"Cui, J., Huo, J., Yang, M.: Novel method of calibration with restrictive constraints for stereo-vision system. J. Mod. Opt. 63(9), 835\u2013846 (2016). https:\/\/doi.org\/10.1080\/09500340.2015.1106602","journal-title":"J. Mod. Opt."},{"key":"263_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijleo.2020.165890","author":"X Feng","year":"2021","unstructured":"Feng, X., Fang, B.: Algorithm for epipolar geometry and correcting monocular stereo vision based on a plane mirror. Optik (2021). https:\/\/doi.org\/10.1016\/j.ijleo.2020.165890","journal-title":"Optik"},{"issue":"6\u20137","key":"263_CR6","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1002\/rob.20293","volume":"26","author":"J Jin","year":"2009","unstructured":"Jin, J., Tang, L.: Corn plant sensing using real-time stereo vision. J. Field Robot. 26(6\u20137), 591\u2013608 (2009). https:\/\/doi.org\/10.1002\/rob.20293","journal-title":"J. Field Robot."},{"issue":"2","key":"263_CR7","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1109\/TRO.2007.914003","volume":"24","author":"C Lightcap","year":"2008","unstructured":"Lightcap, C., Hamner, S., Schmits, T., Banks, S.: Improved positioning accuracy of the PA10-6CE robot with geometric and flexibility calibration. IEEE Trans. Rob. 24(2), 452\u2013456 (2008). https:\/\/doi.org\/10.1109\/TRO.2007.914003","journal-title":"IEEE Trans. Rob."},{"issue":"3","key":"263_CR8","first-page":"539","volume":"58","author":"JZ Liu","year":"2015","unstructured":"Liu, J.Z., Li, P.P., Mao, H.P.: Mechanical and kinematic modeling of assistant vacuum sucking and pulling operation of tomato fruits in robotic harvesting. Trans. ASABE 58(3), 539\u2013550 (2015)","journal-title":"Trans. ASABE"},{"key":"263_CR9","doi-asserted-by":"publisher","unstructured":"Matsumoto, Y. and Zelinsky, A.: An algorithm for real-time stereo vision implementation of head pose and gaze direction measurement. In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580), 28\u201330 March 2000 (2000) https:\/\/doi.org\/10.1109\/AFGR.2000.840680","DOI":"10.1109\/AFGR.2000.840680"},{"issue":"9","key":"263_CR10","doi-asserted-by":"publisher","first-page":"1155","DOI":"10.1080\/00207720010024276","volume":"32","author":"Q Memon","year":"2010","unstructured":"Memon, Q., Khan, S.: Camera calibration and three-dimensional world reconstruction of stereo-vision using neural networks. Int. J. Syst. Sci. 32(9), 1155\u20131159 (2010). https:\/\/doi.org\/10.1080\/00207720010024276","journal-title":"Int. J. Syst. Sci."},{"key":"263_CR11","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0067700","volume-title":"Numerical analysis Lecture Notes in Mathematics","author":"JJ Mor\u00e9","year":"1987","unstructured":"Mor\u00e9, J.J.: The Levenberg\u2013Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Numerical analysis Lecture Notes in Mathematics, vol. 630. Springer, Berlin (1987). https:\/\/doi.org\/10.1007\/BFb0067700"},{"issue":"6","key":"263_CR12","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1016\/S0736-5845(01)00024-2","volume":"17","author":"J Motta","year":"2001","unstructured":"Motta, J., de Carvaho, G., McMaster, R.S.: Robot calibration using a 3D vision-based measurement system with a single camera. Robot. Comput.-Integr. Manuf. 17(6), 487\u2013497 (2001). https:\/\/doi.org\/10.1016\/S0736-5845(01)00024-2","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"263_CR13","unstructured":"NZ MPI. MPI helps attract people to pick green gold on kiwifruit orchards. https:\/\/www.mpi.govt.nz\/news\/media-releases\/mpi-helps-attract-people-to-pick-green-gold-on-kiwifruit-orchards\/. (2021)"},{"key":"263_CR14","unstructured":"NZ MSD. Bay of Plenty Labour Shortage Declaration. Retrieved September 24, 2018, from https:\/\/www.msd.govt.nz\/about-msd-and-our-work\/newsroom\/media-releases\/2018\/bay-of-plenty-labour-shortage-declaration.html. (2018)"},{"key":"263_CR15","first-page":"58","volume":"7","author":"L Mu","year":"2020","unstructured":"Mu, L., Cui, G., Liu, Y., Cui, Y., Fu, L.: Design and simulation of an integrated end-effector for picking kiwifruit by robot. Inf Process Agric 7, 58\u201371 (2020)","journal-title":"Inf Process Agric"},{"issue":"1","key":"263_CR16","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1016\/j.rcim.2012.06.004","volume":"29","author":"A Nubiola","year":"2013","unstructured":"Nubiola, A., Bonev, I.A.: Absolute calibration of an ABB IRB 1600 robot using a laser tracker. Robot. Comput.-Integr. Manuf. 29(1), 236\u2013245 (2013). https:\/\/doi.org\/10.1016\/j.rcim.2012.06.004","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"263_CR17","unstructured":"NZKGI. New Zealand Kiwifruit Labour Shortage. https:\/\/nzkgi.org.nz\/wp-content\/uploads\/2018\/07\/NZ-Kiwifruit-Labour-Shortage-July-2018.pdf. (2018)"},{"key":"263_CR18","unstructured":"NZKGI. NZKGI Labour Report for the 2020 Season. https:\/\/www.nzkgi.org.nz\/wp-content\/uploads\/2021\/10\/NZKGI-Labour-Report-for-the-2020-season.pdf#article. (2020)"},{"key":"263_CR19","doi-asserted-by":"publisher","unstructured":"Park, K., Park, C. and Shin, Y.: Performance Evaluation of Industrial Dual-Arm Robot. 2008 International Conference on Smart Manufacturing Application. (2008) https:\/\/doi.org\/10.1109\/ICSMA.2008.4505596","DOI":"10.1109\/ICSMA.2008.4505596"},{"issue":"4","key":"263_CR20","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1016\/j.rcim.2010.12.009","volume":"27","author":"CR Rocha","year":"2011","unstructured":"Rocha, C.R., Tonetto, C.P., Dias, A.: A comparison between the Denavit\u2013Hartenberg and the screw-based methods used in kinematic modeling of robot manipulators. Robot. Comput.-Integr. Manuf. 27(4), 723\u2013728 (2011). https:\/\/doi.org\/10.1016\/j.rcim.2010.12.009","journal-title":"Robot. Comput.-Integr. Manuf."},{"issue":"6","key":"263_CR21","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1002\/rob.21715","volume":"34","author":"A Silwal","year":"2017","unstructured":"Silwal, A., Davidson, J.R., Karkee, M., Mo, C., Zhang, Q., Lewis, K.: Design, integration, and field evaluation of a robotic apple harvester. J. Field Robot. 34(6), 1140\u20131159 (2017)","journal-title":"J. Field Robot."},{"key":"263_CR22","unstructured":"Statistics New Zealand. Minimum wage pushes up private sector pay rates. Retrieved September 24, 2018, from https:\/\/www.stats.govt.nz\/news\/minimum-wage-pushes-up-private-sector-pay-rates. (2018)"},{"issue":"4","key":"263_CR23","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1109\/JRA.1987.1087109","volume":"3","author":"R Tsai","year":"1987","unstructured":"Tsai, R.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses. IEEE J. Robot. Autom. 3(4), 323\u2013344 (1987)","journal-title":"IEEE J. Robot. Autom."},{"issue":"10","key":"263_CR24","doi-asserted-by":"publisher","first-page":"965","DOI":"10.1109\/34.159901","volume":"14","author":"J Weng","year":"1992","unstructured":"Weng, J., Cohen, P., Hemiou, M.: Camera calibration with distortion models and accuracy evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 965\u2013980 (1992). https:\/\/doi.org\/10.1109\/34.159901","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"263_CR25","doi-asserted-by":"publisher","first-page":"140","DOI":"10.1016\/j.biosystemseng.2019.03.007","volume":"181","author":"HA Williams","year":"2019","unstructured":"Williams, H.A., Jones, M.H., Nejati, M., Seabright, M.J., Bell, J., Penhall, N.D., Barnett, J.J., Duke, M.D., Scarfe, A.J., Ho, S.A., Lim, J.Y., MacDonald, B.A.: Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms. Biosys. Eng. 181, 140\u2013156 (2019). https:\/\/doi.org\/10.1016\/j.biosystemseng.2019.03.007","journal-title":"Biosys. Eng."},{"issue":"2","key":"263_CR26","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1002\/rob.21890","volume":"37","author":"HA Williams","year":"2020","unstructured":"Williams, H.A., Ting, C., Nejati, M., Jones, M.H., Penhall, N., Lim, J.Y., Seabright, M., Bell, J., Ahn, H.S., Scarfe, A., Duke, M., MacDonald, B.A.: Improvements to and large-scale evaluation of a robotic kiwifruit harvester. J. Field Robot. 37(2), 187\u2013201 (2020). https:\/\/doi.org\/10.1002\/rob.21890","journal-title":"J. Field Robot."},{"key":"263_CR27","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1002\/rob.21889","volume":"37","author":"Y Xiong","year":"2020","unstructured":"Xiong, Y., Ge, Y., Grimstad, L., From, P.J.: An autonomous strawberry-harvesting robot: design, development, integration, and field evaluation. J. Field Robot. 37, 202\u2013224 (2020). https:\/\/doi.org\/10.1002\/rob.21889","journal-title":"J. Field Robot."},{"key":"263_CR28","doi-asserted-by":"publisher","unstructured":"Xu, W. and Mills, J. K.: A new approach to the position and orientation calibration of robots. In Proceedings of the 1999 IEEE International Symposium on Assembly and Task Planning (ISATP'99) (Cat. No.99TH8470) (1999) https:\/\/doi.org\/10.1109\/ISATP.1999.782970","DOI":"10.1109\/ISATP.1999.782970"},{"key":"263_CR29","unstructured":"Zespri. Zespri Annual Report 2020\/21. https:\/\/www.zespri.com\/content\/dam\/zespri\/nz\/annual-reports\/Annual_Report_2020_21.pdf. (2020)"},{"key":"263_CR30","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1023\/A:1007941100561","volume":"27","author":"Z Zhang","year":"1998","unstructured":"Zhang, Z.: Determining the epipolar geometry and its uncertainty: a review. Int. J. Comput. Vision 27, 161\u2013195 (1998). https:\/\/doi.org\/10.1023\/A:1007941100561","journal-title":"Int. J. Comput. Vision"},{"key":"263_CR31","doi-asserted-by":"publisher","DOI":"10.1201\/b13856","volume-title":"Handbook of 3D Machine Vision, Optical Metrology and Imaging","author":"S Zhang","year":"2016","unstructured":"Zhang, S.: Handbook of 3D Machine Vision, Optical Metrology and Imaging. CRC Press, Boca Raton (2016)"},{"issue":"3","key":"263_CR32","doi-asserted-by":"publisher","first-page":"316","DOI":"10.1016\/S1000-9361(08)60105-3","volume":"22","author":"H Zhang","year":"2009","unstructured":"Zhang, H., Zhang, L., Wang, H., Chen, J.: Surface measurement based on instantaneous random illumination. Chin. J. Aeronaut. 22(3), 316\u2013324 (2009). https:\/\/doi.org\/10.1016\/S1000-9361(08)60105-3","journal-title":"Chin. J. Aeronaut."}],"container-title":["International Journal of Intelligent Robotics and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41315-022-00263-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41315-022-00263-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41315-022-00263-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,31]],"date-time":"2023-05-31T04:31:39Z","timestamp":1685507499000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41315-022-00263-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,17]]},"references-count":32,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["263"],"URL":"https:\/\/doi.org\/10.1007\/s41315-022-00263-x","relation":{},"ISSN":["2366-5971","2366-598X"],"issn-type":[{"type":"print","value":"2366-5971"},{"type":"electronic","value":"2366-598X"}],"subject":[],"published":{"date-parts":[[2022,10,17]]},"assertion":[{"value":"28 May 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 September 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This study does not require ethics approval as no human or animal subjects are involved.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"This study does not require consent to participate as no human research participant is included in the study.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Authors consent to publish the here presented work.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to publish"}}]}}