Abstract
In the medical domain, where a misdiagnosis can have life-altering ramifications, understanding the certainty of model predictions is an important part of the model development process. However, deep learning approaches suffer from a lack of a native uncertainty metric found in other statistical learning methods. One common technique for uncertainty estimation is the use of Monte-Carlo (MC) dropout at training and inference. Another approach is Conformal Prediction for Uncertainty Quantification (CUQ). This paper will explore these two methods as applied to a cervical cancer screening algorithm currently under development for use in low-resource settings. We find that overall, CUQ and MC inference produce similar uncertainty patterns, that CUQ can aid in model development through class delineation, and that CUQ uncertainty is higher when the model is incorrect, providing further fine-grained information for clinical decisions. Code available here
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmed, S.R., et al.: Reproducible and clinically translatable deep neural networks for cervical screening. Sci. Rep. 13(1), 21772 (2023)
Angelopoulos, A.N., Bates, S.: A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511 (2021)
Camarasa, R., et al.: Quantitative comparison of Monte-Carlo dropout uncertainty measures for multi-class segmentation. In: Sudre, C.H., et al. (eds.) UNSURE/GRAIL -2020. LNCS, vol. 12443, pp. 32–41. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60365-6_4
Catarino, R., Schäfer, S., Vassilakos, P., Petignat, P., Arbyn, M.: Accuracy of combinations of visual inspection using acetic acid or lugol iodine to detect cervical precancer: a meta-analysis. BJOG Int. J. Obstet. Gynaecol. 125(5), 545–553 (2018)
Desai, K.T., et al.: The development of “automated visual evaluation’’ for cervical cancer screening: the promise and challenges in adapting deep-learning for clinical testing. Int. J. Cancer 150(5), 741–752 (2022)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059. PMLR (2016)
Hara, K., Saitoh, D., Shouno, H.: Analysis of dropout learning regarded as ensemble learning. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) ICANN 2016, Part II. LNCS, vol. 9887, pp. 72–79. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44781-0_9
Liming, H., et al.: An observational study of deep learning and automated evaluation of cervical images for cancer screening. JNCI J. Natl. Cancer Inst. 111(9), 923–932 (2019)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Kitchener, H.C., Castle, P.E., Cox, J.T.: Chapter 7: achievements and limitations of cervical cytology screening. Vaccine 24(suppl 3), S3/63–70 (2006). Accessed 23 Sept 2019
Lemay, A., et al.: Improving the repeatability of deep learning models with Monte Carlo dropout. npj Digit. Med. 5(1), 174 (2022)
Charles, L., Lemay, A., Chang, K., Höbel, K., Kalpathy-Cramer, J.: Fair conformal predictors for applications in medical imaging. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol, 36, pp. 12008–12016 (2022)
Lycke, K.D., et al.: Agreement on lesion presence and location at colposcopy. J. Low. Genit. Tract Dis. 28(1), 37–42 (2024)
Mehrtens, H., Bucher, T., Brinker, T.J.: Pitfalls of conformal predictions for medical image classification. In: Sudre, C.H., Baumgartner, C.F., Dalca, A., Mehta, R., Qin, C., Wells, W.M. (eds.) UNSURE 2023. LNCS, vol. 14291, pp. 198–207. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-44336-7_20
Olsson, H., et al.: Estimating diagnostic uncertainty in artificial intelligence assisted pathology using conformal prediction. Nat. Commun. 13(1), 7761 (2022)
World Health Organization: Cervical Cancer (2024). https://www.who.int/news-room/fact-sheets/detail/cervical-cancer. Accessed 19 June 2024
Pal, A., et al.: Deep metric learning for cervical image classification. IEEE Access 9, 53266–53275 (2021)
Shamsunder, S., Mishra, A., Kumar, A., Beriwal, R., Ahluwalia, C., Das, S.: Diagnostic accuracy of artificial intelligence algorithm incorporated into mobileODT enhanced visual assessment for triaging screen positive women after cervical cancer screening (2022)
Silkensen, S.L., Schiffman, M., Sahasrabuddhe, V., Flanigan, J.S.: Is it time to move beyond visual inspection with acetic acid for cervical cancer screening? (2018)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Wentzensen, N., et al.: Accuracy and efficiency of deep-learning-based automation of dual stain cytology in cervical cancer screening. JNCI J. Natl. Cancer Inst. 113(1), 72–79 (2021)
Xue, Z., et al.: A demonstration of automated visual evaluation of cervical images taken with a smartphone camera. Int. J. Cancer 147(9), 2416–2423 (2020)
Zar, J.H.: Spearman rank correlation. Encyclopedia of Biostatistics 7 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Clark, C. et al. (2025). Conformal Prediction and Monte Carlo Inference for Addressing Uncertainty in Cervical Cancer Screening. In: Sudre, C.H., Mehta, R., Ouyang, C., Qin, C., Rakic, M., Wells, W.M. (eds) Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. UNSURE 2024. Lecture Notes in Computer Science, vol 15167. Springer, Cham. https://doi.org/10.1007/978-3-031-73158-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-73158-7_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73157-0
Online ISBN: 978-3-031-73158-7
eBook Packages: Computer ScienceComputer Science (R0)
