{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T18:20:16Z","timestamp":1732040416655},"reference-count":33,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2021,4,21]],"date-time":"2021-04-21T00:00:00Z","timestamp":1618963200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"The apple target recognition algorithm is one of the core technologies of the apple picking robot. However, most of the existing apple detection algorithms cannot distinguish between the apples that are occluded by tree branches and occluded by other apples. The apples, grasping end-effector and mechanical picking arm of the robot are very likely to be damaged if the algorithm is directly applied to the picking robot. Based on this practical problem, in order to automatically recognize the graspable and ungraspable apples in an apple tree image, a light-weight apple targets detection method was proposed for picking robot using improved YOLOv5s. Firstly, BottleneckCSP module was improved designed to BottleneckCSP-2 module which was used to replace the BottleneckCSP module in backbone architecture of original YOLOv5s network. Secondly, SE module, which belonged to the visual attention mechanism network, was inserted to the proposed improved backbone network. Thirdly, the bonding fusion mode of feature maps, which were inputs to the target detection layer of medium size in the original YOLOv5s network, were improved. Finally, the initial anchor box size of the original network was improved. The experimental results indicated that the graspable apples, which were unoccluded or only occluded by tree leaves, and the ungraspable apples, which were occluded by tree branches or occluded by other fruits, could be identified effectively using the proposed improved network model in this study. Specifically, the recognition recall, precision, mAP and F1 were 91.48%, 83.83%, 86.75% and 87.49%, respectively. The average recognition time was 0.015 s per image. Contrasted with original YOLOv5s, YOLOv3, YOLOv4 and EfficientDet-D0 model, the mAP of the proposed improved YOLOv5s model increased by 5.05%, 14.95%, 4.74% and 6.75% respectively, the size of the model compressed by 9.29%, 94.6%, 94.8% and 15.3% respectively. The average recognition speeds per image of the proposed improved YOLOv5s model were 2.53, 1.13 and 3.53 times of EfficientDet-D0, YOLOv4 and YOLOv3 and model, respectively. The proposed method can provide technical support for the real-time accurate detection of multiple fruit targets for the apple picking robot.<\/jats:p>","DOI":"10.3390\/rs13091619","type":"journal-article","created":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T01:25:10Z","timestamp":1619054710000},"page":"1619","source":"Crossref","is-referenced-by-count":388,"title":["A Real-Time Apple Targets Detection Method for Picking Robot Based on Improved YOLOv5"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-2233-0065","authenticated-orcid":false,"given":"Bin","family":"Yan","sequence":"first","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China"},{"name":"Shannxi Key Laboratory of Apple, Yangling 712100, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9104-0598","authenticated-orcid":false,"given":"Pan","family":"Fan","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China"},{"name":"Shannxi Key Laboratory of Apple, Yangling 712100, China"}]},{"given":"Xiaoyan","family":"Lei","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China"},{"name":"Shannxi Key Laboratory of Apple, Yangling 712100, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-1699-3468","authenticated-orcid":false,"given":"Zhijie","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China"},{"name":"Apple Full Mechanized Scientific Research Base of Ministry of Agriculture and Rural Affairs, Yangling 712100, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-0194-0032","authenticated-orcid":false,"given":"Fuzeng","family":"Yang","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, China"},{"name":"Apple Full Mechanized Scientific Research Base of Ministry of Agriculture and Rural Affairs, Yangling 712100, China"},{"name":"State Key Laboratory of Soil Erosion and Dryland Farming on Loess Plateau, Yangling 712100, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fu, L., Gao, F., Wu, J., Li, R., Karkee, M., and Zhang, Q. (2020). Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review. Comput. Electron. Agric., 177.","DOI":"10.1016\/j.compag.2020.105687"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Igathinathane, C., Li, J., Cen, H., Lu, Y., and Flores, P. (2020). Technology progress in mechanical harvest of fresh market apples. Comput. Electron. Agric., 175.","DOI":"10.1016\/j.compag.2020.105606"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ghosh, I., and Datta Chaudhuri, T. (2020). FEB-Stacking and FEB-DNN models for stock trend prediction: A performance analysis for pre and post Covid-19 periods. Decis. Mak. Appl. Manag. Eng., 4.","DOI":"10.31181\/dmame2104051g"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Malinda, M., and Chen, J. (2020). The forecasting of consumer exchange-traded funds (ETFs) via grey relational analysis (GRA) and artificial neural network (ANN). Empir. Econ., 3.","DOI":"10.1007\/s00181-021-02039-x"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Precup, R., Preitl, S., Petriu, E., Bojan-Dragos, C., Szedlak-Stinean, A., Roman, R., and Hedrea, E. (2020). Model-Based fuzzy control results for networked control systems. Rep. Mech. Eng., 1.","DOI":"10.31181\/rme200101010p"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Mirko, S., Aleksandar, S., and \u0110or\u0111e, S. (2019). ANFIS model for the prediction of generated electricity of photovoltaic modules. Decis. Mak. Appl. Manag. Eng., 2.","DOI":"10.31181\/dmame1901035s"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Messinis, S., and Vosniakos, G. (2020). An agent-based flexible manufacturing system controller with Petri-net enabled algebraic deadlock avoidance. Rep. Mech. Eng., 1.","DOI":"10.31181\/rme200101077m"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Hu, L., Liu, Z., Hu, W., Wang, Y., Tan, J., and Wu, F. (2020). Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network. J. Manuf. Syst., 55.","DOI":"10.1016\/j.jmsy.2020.02.004"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kerkech, M., Hafiane, A., and Canals, R. (2020). VddNet: Vine disease detection network based on multispectral images and depth map. Remote Sens., 12.","DOI":"10.3390\/rs12203305"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Fromm, M., Schubert, M., Castilla, G., Linke, J., and McDermid, G. (2019). Automated detection of conifer seedlings in drone imagery using convolutional neural networks. Remote Sens., 11.","DOI":"10.3390\/rs11212585"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Afzaal, H., Farooque, A.A., Schumann, A.W., Hussain, N., McKenzie-Gopsill, A., Esau, T., Abbas, F., and Acharya, B. (2021). Detection of a potato disease (early blight) using artificial intelligence. Remote Sens., 13.","DOI":"10.3390\/rs13030411"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Abdulridha, J., Ampatzidis, Y., Qureshi, J., and Roberts, P. (2020). Laboratory and UAV-Based identification and classification of tomato yellow leaf curl, bacterial spot, and target spot diseases in tomato utilizing hyperspectral imaging and machine learning. Remote Sens., 12.","DOI":"10.3390\/rs12172732"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Biffi, L.J., Mitishita, E., Liesenberg, V., dos Santos, A.A., Goncalves, D.N., Estrabis, N.V., Silva, J.d.A., Osco, L.P., Ramos, A.P.M., and Centeno, J.A.S. (2021). ATSS deep learning-based approach to detect apple fruits. Remote Sens., 13.","DOI":"10.3390\/rs13010054"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fuentes-Pacheco, J., Torres-Olivares, J., Roman-Rangel, E., Cervantes, S., Juarez-Lopez, P., Hermosillo-Valadez, J., and Rend\u00f3n-Mancha, J.M. (2019). Fig plant segmentation from aerial images using a deep convolutional encoder-decoder network. Remote Sens., 11.","DOI":"10.3390\/rs11101157"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1002\/rob.21902","article-title":"A comparative study of fruit detection and counting methods for yield mapping in apple orchards","volume":"37","author":"Hani","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhang, T., and Zhang, X. (2019). High-Speed ship detection in SAR images based on a grid convolutional neural network. Remote Sens., 11.","DOI":"10.3390\/rs11101206"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Peteinatos, G., Reichel, P., Karouta, J., Andujar, D., and Gerhards, R. (2020). Weed Identification in Maize, Sunflower, and Potatoes with the Aid of Convolutional Neural Networks. Remote Sens., 12.","DOI":"10.3390\/rs12244185"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hoeser, T., and Kuenzer, C. (2020). Object detection and image segmentation with deep learning on earth observation data: A review-part i: Evolution and recent trends. Remote Sens., 12.","DOI":"10.3390\/rs12101667"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bresilla, K., Perulli, G.D., Boini, A., Morandi, B., Grappadelli, L.C., and Manfrini, L. (2019). Single-Shot convolution neural networks for real-time fruit detection within the tree. Front. Plant Sci., 10.","DOI":"10.3389\/fpls.2019.00611"},{"key":"ref_20","first-page":"164","article-title":"Apple positioning based on YOLO deep convolutional neural network for picking robot in complex background","volume":"35","author":"Zhao","year":"2019","journal-title":"Trans. Chin. Soc. Agric. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kang, H., and Chen, C. (2020). Fast implementation of real-time fruit detection in apple orchards using deep learning. Comput. Electron. Agric., 168.","DOI":"10.1016\/j.compag.2019.105108"},{"key":"ref_22","first-page":"156","article-title":"Recognition of apple targets before fruits thinning by robot based on R-FCN deep convolution neural network","volume":"35","author":"Wang","year":"2019","journal-title":"Trans. Chin. Soc. Agric. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.biosystemseng.2020.07.007","article-title":"Faster R-CNN-based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting","volume":"197","author":"Fu","year":"2020","journal-title":"Biosyst. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gao, F., Fu, L., Zhang, X., Majeed, Y., Li, R., Karkee, M., and Zhang, Q. (2020). Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Comput. Electron. Agric., 176.","DOI":"10.1016\/j.compag.2020.105634"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1016\/j.compag.2019.05.016","article-title":"Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities","volume":"162","author":"Vilaplana","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, J., Karkee, M., Zhang, Q., Zhang, X., Yaqoob, M., Fu, L., and Wang, S. (2020). Multi-class object detection using faster R-CNN and estimation of shaking locations for automated shake-and-catch apple harvesting. Comput. Electron. Agric., 173.","DOI":"10.1016\/j.compag.2020.105384"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gene-Mola, J., Sanz-Cortiella, R., Rosell-Polo, J.R., Morros, J.-R., Ruiz-Hidalgo, J., Vilaplana, V., and Gregorio, E. (2020). Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry. Comput. Electron. Agric., 169.","DOI":"10.1016\/j.compag.2019.105165"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kang, H., and Chen, C. (2019). Fruit detection and segmentation for apple harvesting using visual sensor in orchards. Sensors, 19.","DOI":"10.3390\/s19204599"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kang, H., and Chen, C. (2020). Fruit detection, segmentation and 3D visualisation of environments in apple orchards. Comput. Electron. Agric., 171.","DOI":"10.1016\/j.compag.2020.105302"},{"key":"ref_30","first-page":"276","article-title":"Research on the use of YOLOv5 object detection algorithm in mask wearing recognition","volume":"6","author":"Liu","year":"2020","journal-title":"World Sci. Res. J."},{"key":"ref_31","unstructured":"(2020, May 18). ultralytics. yolov5. Available online: https:\/\/github.com\/ultralytics\/yolov5."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"28301","DOI":"10.1007\/s11042-020-09342-2","article-title":"Combining an information-maximization-based attention mechanism and illumination invariance theory for the recognition of green apples in natural scenes","volume":"79","author":"Sun","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2011","DOI":"10.1109\/TPAMI.2019.2913372","article-title":"Squeeze-and-Excitation Networks","volume":"42","author":"Hu","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/9\/1619\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,11]],"date-time":"2024-07-11T14:26:19Z","timestamp":1720707979000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/9\/1619"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,21]]},"references-count":33,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["rs13091619"],"URL":"https:\/\/doi.org\/10.3390\/rs13091619","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,21]]}}}