Abstract
Phenology is an important factor in studying climate change’s effect on plant growth. Recent studies on herbarium specimens have afforded valuable information on plant phenology. The initiatives of herbaria to digitize their collections can extend plant phenological research rapidly by providing online access to significant collections of digitized specimen images. However, they present a major outstanding challenge when extracting reliable data from the specimen sheets. To effectively detect the presence/absence of the reproductive organs such as buds, flowers, and fruits from the specimen images, we developed PhenoDeep, a deep learning approach based on the refined Mask Scoring R-CNN approach. The Mask Scoring R-CNN backbone network was modified by exploiting the advantages of combining ResNet and DenseNet architectures. The experimental results indicate that PhenoDeep can segment the reproductive organs within different specimens, where the precision of PhenoDeep reached 94.1% and recall 94.3%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Piao, S., et al.: Plant phenology and global climate change: current progresses and challenges. Glob. Change Biol. 25, 1922–1940 (2019)
Yalcin, H.: Plant phenology recognition using deep learning: Deep-Pheno. In: 2017 6th International Conference on Agro-Geoinformatics, pp. 1–5 (2017)
Milicevic, M., et al.: Application of deep learning architectures for accurate detection of olive tree flowering phenophase. Remote Sen. 12(13), 2120 (2020)
Borsch, T., et al.: A complete digitization of German herbaria is possible, sensible and should be started now. Res. Ideas Outcomes 6, e50675 (2020)
Sweeney, P.W., et al.: Large-scale digitization of herbarium specimens: development and usage of an automated, high–throughput conveyor system. Taxon 67, 165–178 (2018)
Hedrick, B.P., et al.: Digitization and the future of natural history collections. BioScience 70(3), 243–251 (2020)
Park, D., et al.: Assessing Plant Phenological Patterns in the Eastern United States Over the Last 120 Years, Environmental Data Initiative (2018)
Pearson, K.D., et al.: Machine learning using digitized herbarium specimens to advance phenological research. BioScience 70(7), 610–620 (2020)
Lorieul, T., et al.: Toward a large-scale and deep phenological stage annotation of herbarium specimens: case studies from temperate, tropical, and equatorial floras. Appl. Plant Sci. 7(3), e01233 (2019). PMCID: PMC6426157
Ellwood, et al.: Emerging frontiers in phenological research. Appl. Plant Sci. 7(3), e1234 (2019)
Goëau, H., et al.: A new fine-grained method for automated visual analysis of herbarium specimens: a case study for phenological data extraction. Appl. Plant Sci. 8(6), e11368 (2020)
He, K., et al.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)
Davis, C.C., et al.: A new method for counting reproductive structures in digitized herbarium specimens using mask R-CNN. Front. Plant Sci. 11, 1129 (2020). PMID: 32849691; PMCID: PMC7411132
Huang, Z., et al.: Mask scoring R-CNN. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6402–6411 (2019)
Dutta, A., Zisserman, A.: The VIA annotation software for images, audio and video. In: Proceedings of the 27th ACM International Conference on Multimedia (MM 2019), p. 4, 21–25 October 2019, Nice, France. ACM, New York, NY, USA (2019)
Ren, S., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume, pp. 91–99. MIT Press, Cambridge, MA, USA (2015)
Lin, T., et al.: Feature pyramid networks for object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2017)
Huang, G., et al.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Francisco, M., Ross, G.: maskrcnn-benchmark: fast, modular reference implementation of instance segmentation and object detection algorithms. In: PyTorch (2018)
Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6, 60 (2019)
Lima, D.F., Mello, J.H.F., Lopes, I.T., Forzza, R.C., Goldenberg, R., et al.: Phenological responses to climate change based on a hundred years of herbarium collections of tropical Melastomataceae. PLOS ONE 16(5), e0251360 (2021)
Lee, H.K., Lee, S.J., Kim, M.K., Lee, S.D.: Prediction of plant phenological shift under climate change in South Korea. Sustainability 12, 9276 (2020)
Willis, C.G., et al.: Old plants, new tricks: phenological research using herbarium specimens. Trends Ecol. Evol. 32(7), 531–546 (2017). ISSN 0169–53475
Acknowledgments
This work was part of the MAMUDS project (Management Multimedia Data for Science). It was supported by BMBF, Germany (Project No. 01D16009) and MHESR, Tunisia.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Triki, A., Bouaziz, B., Gaikwad, J., Mahdi, W. (2021). PhenoDeep: A Deep Learning-Based Approach for Detecting Reproductive Organs from Digitized Herbarium Specimen Images. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13108. Springer, Cham. https://doi.org/10.1007/978-3-030-92185-9_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-92185-9_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92184-2
Online ISBN: 978-3-030-92185-9
eBook Packages: Computer ScienceComputer Science (R0)