Abstract
This paper aims to study the segmentation demands of vineyard images using Convolutional Neural Networks (CNNs). To this end, eleven CNN models able to provide semantic segmented images are examined as part of the sensing subsystem of an autonomous agricultural robot. The task is challenging due to the similar color between grapes, leaves and image’s background. Moreover, the lack of controlled lighting conditions results in varying color representation of grapes and leaves. The studied CNN model architectures combine three different feature learning sub-networks, with five meta-architectures for segmentation purposes. Investigation on three different datasets consisting of vineyard images of grape clusters and leaves, provided segmentation results, by mean pixel intersection over union (IU) performance index, of up to 87.89% for grape clusters and 83.45% for leaves, for the case of ResNet50_FRRN and MobileNetV2_PSPNet model, respectively. Comparative results reveal the efficacy of CNNs to separate grape clusters and leaves from image’s background. Thus, the proposed models can be used for in-field applications for real-time localization of grapes and leaves, towards automation of harvest, green harvest and defoliation agricultural activities by an autonomous robot.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Xue, J., Zhang, L., Grift, T.E.: Variable field-of-view machine vision based row guidance of an agricultural robot. Comput. Electron. Agric. 84, 85–91 (2012). https://doi.org/10.1016/j.compag.2012.02.009
Søgaard, H.T., Lund, I.: Application accuracy of a machine vision-controlled robotic micro-dosing system. Biosyst. Eng. 96, 315–322 (2007). https://doi.org/10.1016/j.biosystemseng.2006.11.009
Mavridou, E., Vrochidou, E., Papakostas, G.A., Pachidis, T., Kaburlasos, V.G.: Machine vision systems in precision agriculture for crop farming. J. Imaging 5, 89 (2019). https://doi.org/10.3390/jimaging5120089
LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp. 253–256. IEEE (2010). https://doi.org/10.1109/ISCAS.2010.5537907
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440. IEEE (2015). https://doi.org/10.1109/CVPR.2015.7298965
Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: Proceedings of the British Machine Vision Conference 2014, pp. 6.1–6.12. British Machine Vision Association (2014). https://doi.org/10.5244/C.28.6
Szegedy, C., Wei Liu, Yangqing Jia, Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9. IEEE (2015). https://doi.org/10.1109/CVPR.2015.7298594
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017). https://doi.org/10.1145/3065386
Badeka, E., Kalabokas, T., Tziridis, K., Nicolaou, A., Vrochidou, E., Mavridou, E., Papakostas, G.A., Pachidis, T.: Grapes visual segmentation for harvesting robots using local texture descriptors. In: 12th International Conference on Computer Vision Systems (ICVS 2019), pp. 98–109 (2019). https://doi.org/10.1007/978-3-030-34995-0_9
Rudolph, R., Herzog, K., Töpfer, R., Steinhage, V.: Efficient identification, localization and quantification of grapevine inflorescences and flowers in unprepared field images using fully convolutional networks. Vitis J. Grapevine Res. 58(3), 95–104 (2019). https://doi.org/10.5073/vitis.2019.58.95-104
Mei, S., Ji, J., Bi, Q., Hou, J., Du, Q., Li, W.: Integrating spectral and spatial information into deep convolutional neural networks for hyperspectral classification. In: 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 5067–5070. IEEE (2016). https://doi.org/10.1109/IGARSS.2016.7730321
Kangune, K., Kulkarni, V., Kosamkar, P.: Grapes ripeness estimation using convolutional neural network and support vector machine. In: 2019 Global Conference for Advancement in Technology (GCAT), pp. 1–5. IEEE (2019). https://doi.org/10.1109/GCAT47503.2019.8978341
. Li, N., Wang, C., Zhao, H., Gong, X., Wang, D.: A novel deep convolutional neural network for spectral-spatial classification of hyperspectral data. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. ISPRS Arch. 42, 897–900 (2018). https://doi.org/10.5194/isprs-archives-XLII-3-897-2018
Zhao, L., Li, Q., Zhang, Y., Wang, H., Du, X.: Integrating the continuous wavelet transform and a convolutional neural network to identify vineyard using time series satellite images. Remote Sens. 11, 2641 (2019). https://doi.org/10.3390/rs11222641
Monga, T.: Estimating vineyard grape yield from images. In: Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 339–343 (2018). https://doi.org/10.1007/978-3-319-89656-4_37
Yu, S., Jia, S., Xu, C.: Convolutional neural networks for hyperspectral image classification. Neurocomputing 219, 88–98 (2017). https://doi.org/10.1016/j.neucom.2016.09.010
Personalized Optimal Grape Harvest by Autonomous Robot (POGHAR). http://evtar.eu/
Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: Labelme: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77, 157–173 (2008). https://doi.org/10.1007/s11263-007-0090-8
Wong, S.C., Gatt, A., Stamatescu, V., McDonnell, M.D.: Understanding data augmentation for classification: when to warp? In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6. IEEE (2016). https://doi.org/10.1109/DICTA.2016.7797091
Mikolajczyk, A., Grochowski, M.: Data augmentation for improving deep learning in image classification problem. In: 2018 International Interdisciplinary Ph.D. Workshop (IIPhDW), pp. 117–122. IEEE (2018). https://doi.org/10.1109/IIPHDW.2018.8388338
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414–2423. IEEE (2016). https://doi.org/10.1109/CVPR.2016.265
Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition. In: 3rd International Conference Learning Representations, ICLR 2015, Conference Track Proceedings (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE (2016). https://doi.org/10.1109/CVPR.2016.90
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510–4520. IEEE (2018). https://doi.org/10.1109/CVPR.2018.00474
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944. IEEE (2017). https://doi.org/10.1109/CVPR.2017.106
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 234–241 (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6230–6239. IEEE (2017). https://doi.org/10.1109/CVPR.2017.660
Pohlen, T., Hermans, A., Mathias, M., Leibe, B.: Full-resolution residual networks for semantic segmentation in street scenes. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3309–3318. IEEE (2017). https://doi.org/10.1109/CVPR.2017.353
Siam, M., Gamal, M., Abdel-Razek, M., Yogamani, S., Jagersand, M.: RTseg: real-time semantic segmentation comparative study. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 1603–1607. IEEE (2018). https://doi.org/10.1109/ICIP.2018.8451495
Zeiler, M.D.: ADADELTA: an adaptive learning rate method (2012)
Acknowledgment
This research has been co-financed by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH – CREATE – INNOVATE (project code: T1EDK-00300).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kalampokas, T. et al. (2020). Semantic Segmentation of Vineyard Images Using Convolutional Neural Networks. In: Iliadis, L., Angelov, P., Jayne, C., Pimenidis, E. (eds) Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference. EANN 2020. Proceedings of the International Neural Networks Society, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-030-48791-1_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-48791-1_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-48790-4
Online ISBN: 978-3-030-48791-1
eBook Packages: Computer ScienceComputer Science (R0)