Validation workflow for a clinical Bayesian network model in multidisciplinary decision making in head and neck oncology treatment | International Journal of Computer Assisted Radiology and Surgery Skip to main content

Advertisement

Springer Nature Link
Log in

Validation workflow for a clinical Bayesian network model in multidisciplinary decision making in head and neck oncology treatment

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Oncological treatment is being increasingly complex, and therefore, decision making in multidisciplinary teams is becoming the key activity in the clinical pathways. The increased complexity is related to the number and variability of possible treatment decisions that may be relevant to a patient. In this paper, we describe validation of a multidisciplinary cancer treatment decision in the clinical domain of head and neck oncology.

Method

Probabilistic graphical models and corresponding inference algorithms, in the form of Bayesian networks, can support complex decision-making processes by providing a mathematically reproducible and transparent advice. The quality of BN-based advice depends on the quality of the model. Therefore, it is vital to validate the model before it is applied in practice.

Results

For an example BN subnetwork of laryngeal cancer with 303 variables, we evaluated 66 patient records. To validate the model on this dataset, a validation workflow was applied in combination with quantitative and qualitative analyses. In the subsequent analyses, we observed four sources of imprecise predictions: incorrect data, incomplete patient data, outvoting relevant observations, and incorrect model. Finally, the four problems were solved by modifying the data and the model.

Conclusion

The presented validation effort is related to the model complexity. For simpler models, the validation workflow is the same, although it may require fewer validation methods. The validation success is related to the model’s well-founded knowledge base. The remaining laryngeal cancer model may disclose additional sources of imprecise predictions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Available free of charge for academic research and teaching use at http://bayesfusion.com/.

References

  1. Chatenoud L, GaravelloW Pagan E, Bertuccio P, Gallus S, La Vecchia C, Negri E, Bosetti C (2016) Laryngeal cancermortality trends in European countries. Int J Cancer 138(4):833–842. doi:10.1002/ijc.29833

    Article  CAS  PubMed  Google Scholar 

  2. Cypko MA, Hirsch D, Koch L, Stoehr M, Strauss G, K D (2015) Web-tool to support medical experts in probabilistic modelling using large Bayesian networks with an example of rhinosinusitis. Stud Health Technol Inform 216:259–263

    PubMed  Google Scholar 

  3. Cypko MA, Stoehr M, Denecke K (2015) Web-based guiding of clinical experts through the modelling of treatment decision models using MEBN with an example of laryngeal cancer. Int J CARS 10(1):163–164

    Google Scholar 

  4. Cypko MA, Stoehr M, Denecke K, Dietz A, Lemke HU (2014) User interaction with MEBNs for large patient-specific treatment decision models with an example for laryngeal cancer. In: Proceeding of the 28th conference for computer assisted radiology and surgery. Fukuoka, Japan

  5. DeGroot MH, Fienberg SE (1983) The comparison and evaluation of forecasters. J Roy Stat Soc. Series D (The Statistician) 32(1/2):12–22

    Google Scholar 

  6. Diez FJ, Mira J, Iturralde E, Zubillaga S (1997) DIAVAL, a Bayesian expert system for echocardiography. Artif Intell Med 10(1):59–73

    Article  CAS  PubMed  Google Scholar 

  7. Druzdzel MJ (1999) GeNIe: a development environment for graphical decision-analytic models. In: Proceedings of the 1999 annual symposium of the American medical informatics association (AMIA-1999), p 1206. Washington, DC

  8. Druzdzel MJ, Oniśko A, Schwartz D, Dowling JN, Wasyluk H (1999) Knowledge engineering for very large decision-analytic medical models. In: Proceedings of the 1999 annual meeting of the American medical informatics association, p 1049. Washington, DC

  9. Ferlay J, Soerjomataram I, Ervik M, Dikshit R, Eser S, Mathers C, Rebelo M, Parkin DM, Forman D, Bray F (2014) Globocan 2014 v1.0. Technical report on cancer incidence and mortality worldwide: IARC CancerBase No. 11. International Agency for Research on Cancer, Lyon, France

  10. Forastiere A, Weber R, Trotti A (2015) Organ preservation for advanced larynx cancer: issues and outcomes. J Clin Oncol 33(29):3262–3268

    Article  PubMed  PubMed Central  Google Scholar 

  11. Kahneman D, Slovic P, Tversky A (eds) (1982) Judgment under uncertainty: Heuristics and biases. Cambridge University Press, Cambridge, England

  12. Lemke HU, Golubnitschaja O (2014) Towards personal health care with model-guided medicine: long-term PPPM-related strategies and realisation opportunities within ’Horizon 2020’. EPMA J 5(1):1–9

    Article  Google Scholar 

  13. Manoogian J, Benson B (2017) Cognitive bias codex. https://en.wikipedia.org/wiki/List_of_cognitive_biases

  14. Moore AW, Lee MS (1994) Efficient algorithms for minimizing cross validation error. In: 11th international conference on machine learning, pp 842–846. Morgan Kaufmann, San Francisco, California

  15. National Comprehensive Cancer Network (2016) Head and Neck Cancer Cancers. v1. 2016

  16. Oniśko A (2003) Probabilistic causal models in medicine: application to diagnosis of liver disorders. Ph.D. thesis, Institute of Biocybernetics and Biomedical Engineering, Polish Academy of Science, Warsaw

  17. Pearl J (1998) Probabilistic reasoning in intelligent systems. Morgan Kaufmann, Burlington

    Google Scholar 

  18. Pitchforth J, Mengersen K (2013) A proposed validation framework for expert elicited Bayesian networks. Expert Syst Appl 40(1):162–167

    Article  Google Scholar 

  19. Stoehr M, Cypko MA, Denecke K, Lemke HU, Dietz A (2014) A model of the decision-making process: therapy of laryngeal cancer. Int J CARS 9(Suppl 1):217–218

  20. Witten IH, Eibe F (2005) Data mining: practical machine learning tools and techniques. Morgan Kaufmann, Burlington

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank J. Gaebel, Y. Deng, S. Oeltze-Jafra, and A. Oniśko for their valuable comments and suggestions that lead to improvements in the quality of the paper.

Funding ICCAS is funded by the German Federal Ministry of Education and Research (BMBF). The statements made herein are solely the responsibility of the authors.

Author information

Authors and Affiliations

  1. Innovation Center Computer Assisted Surgery, University of Leipzig, Semmelweisstr. 14, 04103, Leipzig, Germany

    Mario A. Cypko & Heinz U. Lemke

  2. Clinic of Otolaryngology, Head and Neck Surgery, Department of Head Medicine and Oral Health, University of Leipzig, Leipzig, Germany

    Matthaeus Stoehr & Andreas Dietz

  3. School of Information Sciences, University of Pittsburgh, Pittsburgh, PA, USA

    Marcin Kozniewski & Marek J. Druzdzel

  4. Faculty of Computer Science Bialystok University of Technology, Bialystok, Poland

    Marcin Kozniewski & Marek J. Druzdzel

  5. New York Methodist Hospital, Brooklyn, NY, USA

    Leonard Berliner

Authors
  1. Mario A. Cypko
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Matthaeus Stoehr
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Marcin Kozniewski
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Marek J. Druzdzel
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Andreas Dietz
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Leonard Berliner
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Heinz U. Lemke
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Mario A. Cypko.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

For this type of study, formal consent is not required. This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This articles does not contain patient information.

Appendix: TNM staging system for the Larynx [15]

Appendix: TNM staging system for the Larynx [15]

Primary tumor (T)

TX

Primary tumor cannot be assessed

T0

No evidence of primary tumor

Tis

Carcinoma in situ

T1

Tumor \( \le \)2 cm in greatest dimension

Supraglottis: Tumor limited to one subsite of supraglottis with normal vocal cord mobility

Glottis: Tumor limited to the vocal cord(s) (may involve anterior or posterior commissure), with normal mobility

Subglottis: Tumor limited to the subglottis

T1a

Glottis: Tumor limited to 1 vocal cord

T1b

Glottis: Tumor involves both vocal cords

T2

Tumor >2 cm but not more than 4 cm in greatest dimension

Supraglottis: Tumor invades mucosa of more than one adjacent subsite of supraglottis or glottis or region outside the supraglottis, without fixation of the larynx

Glottis: Tumor extends to the supraglottis and/or subglottis, and/or with impaired vocal cord mobility

Subglottis: Tumor extends to vocal cord(s), with normal or impaired mobility

T3

Tumor >4 cm in greatest dimension

Supraglottis: Tumor limited to the larynx, with vocal cord fixation, and/or invades any of the following: postcricoid area, preepiglottic space, paraglottic space, and/or inner cortex of the thyroid cartilage

Glottis: Tumor limited to the larynx with vocal cord fixation and/or invasion of the paraglottic space and/or inner cortex of the thyroid cartilage

Subglottis: Tumor limited to the larynx, with vocal cord fixation

T4a

Moderately advanced, local disease

   Lip—Tumor invades through cortical bone, inferior alveolar nerve, floor of mouth, or skin of face

   Oral cavity—Tumor invades adjacent structures

Supraglottis, Glottis and Subglottis: Moderately advanced, local disease

   Tumor invades the outer cortex of the thyroid cartilage or through the thyroid cartilage and/or invades tissues beyond the larynx

T4b

Very advanced, local disease

   Tumor invades masticator space, pterygoid plates, or skull base and/or encases internal carotid artery

Supraglottis, Glottis and Subglottis: Very advanced, local disease

   Tumor invades prevertebral space, encases carotid artery, or invades mediastinal structures

Regional lymph nodes (N)

NX

Regional nodes cannot be assessed

N0

No regional lymph node metastasis

N1

Metastasis in a single ipsilateral lymph node 3 cm in greatest dimension

N2

Metastasis in a single ipsilateral lymph node >3 cm but not more than 6 cm in greatest dimension; or in multiple ipsilateral lymph nodes, none >6 cm in greatest dimension; or in bilateral or contralateral lymph nodes, none >6 cm in greatest dimension

N2a

Metastasis in a single ipsilateral lymph node >3 cm but not more than 6 cm in greatest dimension

N2b

Metastasis in multiple ipsilateral lymph nodes, none >6 cm in greatest dimension

N2c

Metastasis in bilateral or contralateral lymph nodes, none >6 cm in greatest dimension

N3

Metastasis in a lymph node >6 cm in greatest dimension

Distant metastasis (M)

M0

No distant metastasis

M1

Distant metastasis

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cypko, M.A., Stoehr, M., Kozniewski, M. et al. Validation workflow for a clinical Bayesian network model in multidisciplinary decision making in head and neck oncology treatment. Int J CARS 12, 1959–1970 (2017). https://doi.org/10.1007/s11548-017-1531-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-017-1531-7

Keywords