{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,13]],"date-time":"2024-08-13T11:12:44Z","timestamp":1723547564589},"reference-count":37,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T00:00:00Z","timestamp":1636416000000},"content-version":"am","delay-in-days":365,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#am"},{"start":{"date-parts":[[2020,11,9]],"date-time":"2020-11-09T00:00:00Z","timestamp":1604880000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100005825","name":"National Institute of Food and Agriculture","doi-asserted-by":"publisher","award":["1001246","1005200","1005756"],"id":[{"id":"10.13039\/100005825","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Journal of Field Robotics"],"published-print":{"date-parts":[[2021,5]]},"abstract":"Abstract<\/jats:title>Fresh market apples are one of the high\u2010value crops in the United States. Washington alone has produced two\u2010thirds of the annual national production in the past 10 years. However, the availability of seasonal labor is increasingly uncertain. Shake\u2010and\u2010catch automated harvesting solutions have, therefore, become attractive for addressing this challenge. One of the significant challenges in applying this harvesting system is effectively positioning the end\u2010effector at appropriate excitation locations. A computer vision system was used for automatically identifying appropriate locations. Convolutional neural networks (CNNs) were utilized to identify the tree trunks and branches for supporting the automated excitation locations determination. Three CNN architectures were employed: Deeplab v3+ ResNet\u201018, VGG\u201016, and VGG\u201019. Four pixel classes were predefined as branches, trunks, apples, and leaves to segment the canopies trained to simple, narrow, accessible, and productive\u00a0tree architectures with varying foliage density. Results on Fuji cultivar showed that ResNet\u201018 outperformed the VGGs in identifying branches and trunks based on all three evaluation measures: per\u2010class accuracy (PcA), intersection over union (IoU), and boundary\u2010F1 score (BFScore). ResNet\u201018 achieved a PcA of 97%, IoU of 0.69, and BFScore of 0.89. The ResNet\u201018 was further evaluated for its robustness with other test canopy images. When applied this method to one of the highest density cultivars of Scifresh, results showed it can achieve IoUs of 0.41 and 0.62 and BFScores of 0.71 and 0.86 for branches and trunks. Such identification result helped to get a 72% of auto\u2010determined shaking points being the \u201cgood\u201d category identified by human experts.<\/jats:p>","DOI":"10.1002\/rob.21998","type":"journal-article","created":{"date-parts":[[2020,11,9]],"date-time":"2020-11-09T15:38:33Z","timestamp":1604936313000},"page":"476-493","update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":42,"title":["Computer vision\u2010based tree trunk and branch identification and shaking points detection in Dense\u2010Foliage canopy for automated harvesting of apples"],"prefix":"10.1002","volume":"38","author":[{"ORCID":"http:\/\/orcid.org\/0000-0001-9654-3859","authenticated-orcid":false,"given":"Xin","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Biological Systems Engineering Washington State University Pullman WA USA"},{"name":"Center for Precision and Automated Agricultural Systems Washington State University Prosser WA USA"}]},{"given":"Manoj","family":"Karkee","sequence":"additional","affiliation":[{"name":"Department of Biological Systems Engineering Washington State University Pullman WA USA"},{"name":"Center for Precision and Automated Agricultural Systems Washington State University Prosser WA USA"}]},{"given":"Qin","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Biological Systems Engineering Washington State University Pullman WA USA"},{"name":"Center for Precision and Automated Agricultural Systems Washington State University Prosser WA USA"}]},{"given":"Matthew D.","family":"Whiting","sequence":"additional","affiliation":[{"name":"Center for Precision and Automated Agricultural Systems Washington State University Prosser WA USA"},{"name":"Department of Horticulture Washington State University Prosser WA USA"}]}],"member":"311","published-online":{"date-parts":[[2020,11,9]]},"reference":[{"key":"e_1_2_6_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.biosystemseng.2015.10.003"},{"key":"e_1_2_6_3_1","doi-asserted-by":"publisher","DOI":"10.3390\/robotics6040031"},{"key":"e_1_2_6_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21525"},{"key":"e_1_2_6_5_1","first-page":"3626","volume-title":"Paper presented at the IEEE International Conference on Robotics and Automation (ICRA '17)","author":"Bargoti S.","year":"2017"},{"key":"e_1_2_6_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2017.12.001"},{"key":"e_1_2_6_7_1","unstructured":"Chen L. C. Papandreou G. Schroff F. &Adam H.(2017). Rethinking atrous convolution for semantic image segmentation.https:\/\/arxiv.org\/abs\/1706.05587"},{"key":"e_1_2_6_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"e_1_2_6_9_1","unstructured":"Clark M.(2017). Washington state's agricultural labor shortage.https:\/\/www.washingtonpolicy.org\/library\/doclib\/Clark-Washington-state-s-agricultural-labor-shortage-PB-6-23-17.pdf"},{"key":"e_1_2_6_10_1","doi-asserted-by":"crossref","unstructured":"Csurka G. Larlus D. Perronnin F. &Meylan F.(2013).What is a good evaluation measure for semantic segmentation?Paper presented at the 24th British Machine Vision Conference (BMVC '13) Bristol U.K (p. 27).https:\/\/doi.org\/10.5244\/C.27.32","DOI":"10.5244\/C.27.32"},{"key":"e_1_2_6_11_1","doi-asserted-by":"crossref","unstructured":"Deng J. Dong W. Socher R. Li L. Li K. &Li F.(2009).Imagenet: A large\u2010scale hierarchical image database. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '09) Miami FL(pp.248\u2013255).https:\/\/doi.org\/10.1109\/CVPR.2009.5206848","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_6_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2849498"},{"key":"e_1_2_6_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.biosystemseng.2020.07.007"},{"key":"e_1_2_6_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2020.105634"},{"key":"e_1_2_6_15_1","doi-asserted-by":"crossref","unstructured":"He K. Gkioxari G. Doll\u00e1r P. &Girshick R.(2017).Mask R\u2010CNN. Paper presented at the IEEE International Conference on Computer Vision (ICCV '17) Venice Italy (pp.2961\u20132969).https:\/\/doi.org\/10.1109\/ICCV.2017.322","DOI":"10.1109\/ICCV.2017.322"},{"key":"e_1_2_6_16_1","doi-asserted-by":"crossref","unstructured":"He K. Zhang X. Ren S. &Sun J.(2016).Identity mappings in deep residual networks. Paper presented at the European Conference on Computer Vision (ECCV '16) Amsterdam Netherlands (pp.630\u2013645).https:\/\/doi.org\/10.1007\/978-3-319-46493-0_38","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"e_1_2_6_17_1","doi-asserted-by":"publisher","DOI":"10.13031\/aea.12974"},{"key":"e_1_2_6_18_1","doi-asserted-by":"publisher","DOI":"10.13031\/trans.12986"},{"key":"e_1_2_6_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2018.02.016"},{"issue":"3","key":"e_1_2_6_20_1","first-page":"565","article-title":"A method for three\u2010dimensional reconstruction of apple trees for automated pruning","volume":"58","author":"Karkee M.","year":"2015","journal-title":"Transactions of the ASABE"},{"key":"e_1_2_6_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2014.02.013"},{"key":"e_1_2_6_22_1","unstructured":"Kemker R. Salvaggio C. &Kanan C.(2017). High\u2010resolution multispectral dataset for semantic segmentation.https:\/\/arxiv.org\/abs\/1703.01918"},{"key":"e_1_2_6_23_1","unstructured":"Krizhevsky A. Sutskever I. &Hinton G. E.(2012).Imagenet classification with deep convolutional neural networks. Paper presented at the Neural Information Processing Systems (NIPS '12) Lake Tahoe CA (pp.1097\u20131105).http:\/\/papers.nips.cc\/paper\/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf"},{"key":"e_1_2_6_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2020.105277"},{"key":"e_1_2_6_25_1","volume-title":"Machine learning: A probabilistic perspective","author":"Murphy K. P.","year":"2012"},{"key":"e_1_2_6_26_1","doi-asserted-by":"publisher","DOI":"10.13031\/2013.13266"},{"issue":"5","key":"e_1_2_6_27_1","first-page":"545","article-title":"Fresh\u2010market quality tree fruit harvester part II: Apples","volume":"19","author":"Peterson D. L.","year":"2003","journal-title":"Applied Engineering in Agriculture"},{"key":"e_1_2_6_28_1","unstructured":"Ren S. He K. Girshick R. &Sun J.(2015).Faster R\u2010CNN: Towards real\u2010time object detection with region proposal networks. Paper presented at the Neural Information Processing Systems (NIPS '15) Montr\u00e9al Canada (pp.91\u201399).http:\/\/papers.nips.cc\/paper\/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks.pdf"},{"key":"e_1_2_6_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2017.2774979"},{"key":"e_1_2_6_30_1","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21715"},{"key":"e_1_2_6_31_1","unstructured":"Simonyan K. &Zisserman A.(2014). Very deep convolutional networks for large\u2010scale image recognition.https:\/\/arxiv.org\/abs\/1409.1556"},{"key":"e_1_2_6_32_1","unstructured":"USDA. (2002). S51.300: United States standards for grades of apples. Washington DC: USDA Agricultural Marketing Service.https:\/\/www.ams.usda.gov\/grades-standards\/applegrades-standards"},{"key":"e_1_2_6_33_1","volume-title":"National agricultural statistics database","author":"USDA","year":"2020"},{"key":"e_1_2_6_34_1","first-page":"93","volume-title":"Automation in Tree Fruit Production: Principles and Practice","author":"Whiting M. D.","year":"2018"},{"key":"e_1_2_6_35_1","doi-asserted-by":"crossref","unstructured":"Zabawa L. Kicherer A. Klingbeil L. Milioto A. Topfer R. Kuhlmann H. &Roscher R.(2019).Detection of single grapevine berries in images using fully convolutional neural networks. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '19) Workshops Long Beach CA.http:\/\/openaccess.thecvf.com\/content_CVPRW_2019\/papers\/CVPPP\/Zabawa_Detection_of_Single_Grapevine_Berries_in_Images_Using_Fully_Convolutional_CVPRW_2019_paper.pdf","DOI":"10.1109\/CVPRW.2019.00313"},{"key":"e_1_2_6_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2018.10.029"},{"key":"e_1_2_6_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2020.105384"},{"key":"e_1_2_6_38_1","doi-asserted-by":"publisher","DOI":"10.13031\/trans.13779"}],"container-title":["Journal of Field Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/rob.21998","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/rob.21998","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/am-pdf\/10.1002\/rob.21998","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/rob.21998","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T06:07:28Z","timestamp":1693548448000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rob.21998"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,9]]},"references-count":37,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,5]]}},"alternative-id":["10.1002\/rob.21998"],"URL":"https:\/\/doi.org\/10.1002\/rob.21998","archive":["Portico"],"relation":{},"ISSN":["1556-4959","1556-4967"],"issn-type":[{"value":"1556-4959","type":"print"},{"value":"1556-4967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,9]]},"assertion":[{"value":"2020-05-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-10-29","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}