{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:53:43Z","timestamp":1740149623355,"version":"3.37.3"},"reference-count":67,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T00:00:00Z","timestamp":1683849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"In the realm of computer vision, semantic segmentation is the task of recognizing objects in images at the pixel level. This is done by performing a classification of each pixel. The task is complex and requires sophisticated skills and knowledge about the context to identify objects\u2019 boundaries. The importance of semantic segmentation in many domains is undisputed. In medical diagnostics, it simplifies the early detection of pathologies, thus mitigating the possible consequences. In this work, we provide a review of the literature on deep ensemble learning models for polyp segmentation and develop new ensembles based on convolutional neural networks and transformers. The development of an effective ensemble entails ensuring diversity between its components. To this end, we combined different models (HarDNet-MSEG, Polyp-PVT, and HSNet) trained with different data augmentation techniques, optimization methods, and learning rates, which we experimentally demonstrate to be useful to form a better ensemble. Most importantly, we introduce a new method to obtain the segmentation mask by averaging intermediate masks after the sigmoid layer. In our extensive experimental evaluation, the average performance of the proposed ensembles over five prominent datasets beat any other solution that we know of. Furthermore, the ensembles also performed better than the state-of-the-art on two of the five datasets, when individually considered, without having been specifically trained for them.<\/jats:p>","DOI":"10.3390\/s23104688","type":"journal-article","created":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T13:56:18Z","timestamp":1683899778000},"page":"4688","source":"Crossref","is-referenced-by-count":9,"title":["Ensembles of Convolutional Neural Networks and Transformers for Polyp Segmentation"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3502-7209","authenticated-orcid":false,"given":"Loris","family":"Nanni","sequence":"first","affiliation":[{"name":"Department of Information Engineering, University of Padova, 35122 Padova, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3210-4632","authenticated-orcid":false,"given":"Carlo","family":"Fantozzi","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Padova, 35122 Padova, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9846-0157","authenticated-orcid":false,"given":"Andrea","family":"Loreggia","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Brescia, 25121 Brescia, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0290-7354","authenticated-orcid":false,"given":"Alessandra","family":"Lumini","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Bologna, 40126 Bologna, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"145","DOI":"10.3322\/caac.21601","article-title":"Colorectal cancer statistics, 2020","volume":"70","author":"Siegel","year":"2020","journal-title":"CA Cancer J. Clin."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4667","DOI":"10.18203\/2320-6012.ijrms20174914","article-title":"Colorectal cancer: A review","volume":"5","author":"Hazzel","year":"2017","journal-title":"Artic. Int. J. Res. Med. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1016\/j.bpg.2017.07.002","article-title":"Adenoma detection rate and risk of colorectal cancer","volume":"31","author":"Wieszczy","year":"2017","journal-title":"Best Pract. Res. Clin. Gastroenterol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1007\/s11548-013-0926-3","article-title":"Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer","volume":"9","author":"Silva","year":"2014","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1488","DOI":"10.1109\/TMI.2014.2314959","article-title":"Automated polyp detection in colon capsule endoscopy","volume":"33","author":"Mamonov","year":"2014","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"630","DOI":"10.1109\/TMI.2015.2487997","article-title":"Automated polyp detection in colonoscopy videos using shape and context information","volume":"35","author":"Tajbakhsh","year":"2016","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3166","DOI":"10.1016\/j.patcog.2012.03.002","article-title":"Towards automatic polyp detection with a polyp appearance model","volume":"45","author":"Bernal","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_8","first-page":"3523","article-title":"Image Segmentation Using Deep Learning: A Survey","volume":"44","author":"Minaee","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Jha, D., Smedsrud, P.H., Riegler, M.A., Johansen, D., Lange, T.D., Halvorsen, P., and Johansen, H.D. (2019, January 9\u201311). ResUNet++: An Advanced Architecture for Medical Image Segmentation. Proceedings of the 2019 IEEE International Symposium on Multimedia (ISM), San Diego, CA, USA.","DOI":"10.1109\/ISM46123.2019.00049"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., and Joskowicz, L. (2020). Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2020, Springer.","DOI":"10.1007\/978-3-030-59716-0"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., and Essert, C. (2021). Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2021, Springer.","DOI":"10.1007\/978-3-030-87193-2"},{"key":"ref_12","unstructured":"Huang, C.H., Wu, H.Y., and Lin, Y.L. (2021). HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS. arXiv."},{"key":"ref_13","unstructured":"Dong, B., Wang, W., Fan, D.P., Li, J., Fu, H., and Shao, L. (2021). Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"106173","DOI":"10.1016\/j.compbiomed.2022.106173","article-title":"HSNet: A hybrid semantic network for polyp segmentation","volume":"150","author":"Zhang","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref_15","first-page":"2","article-title":"Voting with random classifiers (VORACE): Theoretical and experimental analysis","volume":"35","author":"Cornelio","year":"2021","journal-title":"Auton. Agent"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"104003","DOI":"10.1016\/j.compbiomed.2020.104003","article-title":"A comprehensive review of deep learning in colon cancer","volume":"126","author":"Pacal","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3997","DOI":"10.1109\/TIP.2022.3177129","article-title":"Underwater Image Enhancement via Minimal Color Loss and Locally Adaptive Contrast Enhancement","volume":"31","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"103465","DOI":"10.1016\/j.bspc.2021.103465","article-title":"Automated colorectal polyp detection based on image enhancement and dual-path CNN architecture","volume":"73","author":"Nisha","year":"2022","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Springer.","DOI":"10.1007\/978-3-319-24553-9"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"5666","DOI":"10.1002\/mp.13865","article-title":"Automated polyp segmentation for colonoscopy images: A method based on convolutional neural networks and ensemble learning","volume":"46","author":"Guo","year":"2019","journal-title":"Med. Phys."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1007\/s11633-022-1371-y","article-title":"Video polyp segmentation: A deep learning perspective","volume":"19","author":"Ji","year":"2022","journal-title":"Mach. Intell. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_23","unstructured":"Singh, N.K., and Raza, K. (2021). Health Informatics: A Computational Perspective in Healthcare, Springer."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s11704-019-8208-z","article-title":"A survey on ensemble learning","volume":"14","author":"Dong","year":"2020","journal-title":"Front. Comput. Sci."},{"key":"ref_25","unstructured":"Tomar, N.K., Ibtehaz, N., Jha, D., Halvorsen, P., and Ali, S. (2021, January 13). Improving Generalizability in Polyp Segmentation using Ensemble Convolutional Neural Network. Proceedings of the 3rd International Workshop and Challenge on Computer Vision in Endoscopy (EndoCV 2021) Co-Located with the 18th IEEE International Symposium on Biomedical Imaging (ISBI 2021), Nice, France."},{"key":"ref_26","unstructured":"Tran, T.N., Isensee, F., Kr\u00e4mer, L., Yamlahi, A., Adler, T., Godau, P., Tizabi, M., and Maier-Hein, L. (2022, January 28\u201331). Heterogeneous Model Ensemble For Automatic Polyp Segmentation In Endoscopic Video Sequences. Proceedings of the 4th International Workshop and Challenge on Computer Vision in Endoscopy (EndoCV 2022) Co-Located with the 19th IEEE International Symposium on Biomedical Imaging (ISBI 2022), Kolkata, India."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lumini, A., Nanni, L., and Maguolo, G. (2021). Deep ensembles based on Stochastic Activation Selection for Polyp Segmentation. arXiv.","DOI":"10.20944\/preprints202107.0691.v1"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"820","DOI":"10.3390\/signals2040047","article-title":"Deep Ensembles Based on Stochastic Activations for Semantic Segmentation","volume":"2","author":"Lumini","year":"2021","journal-title":"Signals"},{"key":"ref_29","unstructured":"Nanni, L., Cuza, D., Lumini, A., Loreggia, A., and Brahnam, S. (2021). Deep ensembles in bioimage segmentation. arXiv."},{"key":"ref_30","unstructured":"Nanni, L., Cuza, D., Lumini, A., and Brahnam, S. (2022). Computational Intelligence Based Solutions for Vision Systems, IOP Publishing."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"26440","DOI":"10.1109\/ACCESS.2019.2900672","article-title":"Ensemble of Instance Segmentation Models for Polyp Segmentation in Colonoscopy Images","volume":"7","author":"Kang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_32","unstructured":"Shrestha, S., Khanal, B., and Ali, S. (2020, January 14\u201315). Ensemble U-Net Model for Efficient Polyp Segmentation. Proceedings of the MediaEval 2020 Workshop, Online."},{"key":"ref_33","unstructured":"Hong, A., Lee, G., Lee, H., Seo, J., and Yeo, D. (2021, January 13). Deep Learning Model Generalization with Ensemble in Endoscopic Images. Proceedings of the 3rd International Workshop and Challenge on Computer Vision in Endoscopy (EndoCV 2021) Co-Located with the 18th IEEE International Symposium on Biomedical Imaging (ISBI 2021), Nice, France."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"33795","DOI":"10.1109\/ACCESS.2019.2904094","article-title":"Robust Boundary Segmentation in Medical Images Using a Consecutive Deep Encoder-Decoder Network","volume":"7","author":"Nguyen","year":"2019","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Thu Hong, L.T., Chi Thanh, N., and Long, T.Q. (2020, January 14\u201315). Polyp Segmentation in Colonoscopy Images Using Ensembles of U-Nets with EfficientNet and Asymmetric Similarity Loss Function. Proceedings of the 2020 RIVF International Conference on Computing and Communication Technologies (RIVF), Ho Chi Minh City, Vietnam.","DOI":"10.1109\/RIVF48685.2020.9140793"},{"key":"ref_36","unstructured":"Thambawita, V., Hicks, S., Halvorsen, P., and Riegler, M. (2021, January 13). DivergentNets: Medical Image Segmentation by Network Ensemble. Proceedings of the 3rd International Workshop and Challenge on Computer Vision in Endoscopy (EndoCV 2021) Co-Located with the 18th IEEE International Symposium on Biomedical Imaging (ISBI 2021), Nice, France."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"341","DOI":"10.3390\/signals3020022","article-title":"An Empirical Study on Ensemble of Segmentation Approaches","volume":"3","author":"Nanni","year":"2022","journal-title":"Signals"},{"key":"ref_38","unstructured":"Nanni, L., Cuza, D., Lumini, A., Loreggia, A., and Brahman, S. (2023). Artificial Intelligence and Machine Learning for Healthcare, Springer International Publishing. Vol. 1: Image and Data Analytics."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.compmedimag.2015.02.007","article-title":"WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians","volume":"43","author":"Bernal","year":"2015","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). Computer Vision\u2014ECCV 2018, Springer.","DOI":"10.1007\/978-3-030-01228-1"},{"key":"ref_42","unstructured":"Chaudhuri, K., and Salakhutdinov, R. (2019, January 9\u201315). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ro, Y.M., Cheng, W.H., Kim, J., Chu, W.T., Cui, P., Choi, J.W., Hu, M.C., and De Neve, W. (2020). MultiMedia Modeling, Springer.","DOI":"10.1007\/978-3-030-37734-2"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Goutte, C., and Zhu, X. (2020). Advances in Artificial Intelligence, Springer.","DOI":"10.1007\/978-3-030-47358-7"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., Tavares, J.M.R., Bradley, A., Papa, J.P., and Belagiannis, V. (2018). Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-00889-5"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature Pyramid Networks for Object Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_47","first-page":"4037190","article-title":"A benchmark for endoluminal scene segmentation of colonoscopy images","volume":"2017","author":"Bernal","year":"2017","journal-title":"J. Healthc. Eng."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Baheti, B., Innani, S., Gajre, S., and Talbar, S. (2020, January 14\u201319). Eff-UNet: A Novel Architecture for Semantic Segmentation in Unstructured Environment. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00187"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1038\/s41592-020-01008-z","article-title":"nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation","volume":"18","author":"Isensee","year":"2021","journal-title":"Nat. Methods"},{"key":"ref_50","unstructured":"Tao, A., Sapra, K., and Catanzaro, B. (2020). Hierarchical Multi-Scale Attention for Semantic Segmentation. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Jadon, S. (2020). A survey of loss functions for semantic segmentation. arXiv.","DOI":"10.1109\/CIBCB48159.2020.9277638"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., and Cardoso, M.J. (2017). Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. arXiv.","DOI":"10.1007\/978-3-319-67558-9_28"},{"key":"ref_53","unstructured":"Rahman, M.A., and Wang, Y. (2016). International Symposium on Visual Computing, Springer."},{"key":"ref_54","unstructured":"Cho, Y.J. (2021). Weighted Intersection over Union (wIoU): A New Evaluation Metric for Image Segmentation. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1937","DOI":"10.1007\/s11063-018-09977-1","article-title":"Learning from imbalanced datasets with weighted cross-entropy function","volume":"50","author":"Aurelio","year":"2019","journal-title":"Neural Process. Lett."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., de Lange, T., Johansen, D., Spampinato, C., Dang-Nguyen, D.T., Lux, M., and Schmidt, P.T. (2017, January 20\u201323). Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection. Proceedings of the 8th ACM on Multimedia Systems Conference, Taipei, Taiwan.","DOI":"10.1145\/3083187.3083212"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"16621","DOI":"10.1109\/ACCESS.2023.3244197","article-title":"Medical image segmentation based on Transformer and HarDNet structures","volume":"11","author":"Shen","year":"2023","journal-title":"IEEE Access"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"108824","DOI":"10.1016\/j.knosys.2022.108824","article-title":"MIA-Net: Multi-information aggregation network combining transformers and convolutional feature learning for polyp segmentation","volume":"247","author":"Li","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Wu, Y.H., Liu, Y., Zhan, X., and Cheng, M.M. (IEEE Trans. Pattern Anal. Mach. Intell., 2022). P2T: Pyramid Pooling Transformer for Scene Understanding, IEEE Trans. Pattern Anal. Mach. Intell., early access.","DOI":"10.1109\/TPAMI.2022.3202765"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"106304","DOI":"10.1016\/j.compbiomed.2022.106304","article-title":"DBMF: Dual Branch Multiscale Feature Fusion Network for polyp segmentation","volume":"151","author":"Liu","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H. (2021, January 20\u201325). Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00681"},{"key":"ref_62","unstructured":"Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Kim, T., Lee, H., and Kim, D. (2021, January 20\u201324). UACANet: Uncertainty Augmented Context Attention for Polyp Segmentation. Proceedings of the 29th ACM International Conference on Multimedia\u2014MM \u201921, Virtual, China.","DOI":"10.1145\/3474085.3475375"},{"key":"ref_64","unstructured":"Wei, J., Hu, Y., Zhang, R., Li, Z., Zhou, S.K., and Cui, S. (2021). Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Zhao, X., Zhang, L., and Lu, H. (2021). Automatic Polyp Segmentation via Multi-scale Subtraction Network. arXiv.","DOI":"10.1007\/978-3-030-87193-2_12"},{"key":"ref_66","first-page":"616","article-title":"SwinE-Net: Hybrid deep learning approach to novel polyp segmentation using convolutional neural network and Swin Transformer","volume":"9","author":"Park","year":"2022","journal-title":"J. Comput. Des. Eng."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"105476","DOI":"10.1016\/j.compbiomed.2022.105476","article-title":"Attention based multi-scale parallel network for polyp segmentation","volume":"146","author":"Song","year":"2022","journal-title":"Comput. Biol. Med."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/10\/4688\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T14:40:57Z","timestamp":1683902457000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/10\/4688"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,12]]},"references-count":67,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["s23104688"],"URL":"https:\/\/doi.org\/10.3390\/s23104688","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,5,12]]}}}