Abstract
Large Language Models (LLMs) offer promising solutions for text summarization. However, some domains require specific information to be available in the summaries. Generating these domain-adapted summaries is still an open challenge. Similarly, hallucinations in generated content is a major drawback of current approaches, preventing their deployment. This study proposes a novel approach that leverages ontologies to create domain-adapted summaries both structured and unstructured. We employ an ontology-guided constrained decoding process to reduce hallucinations while improving relevance. When applied to the medical domain, our method shows potential in summarizing Electronic Health Records (EHRs) across different specialties, allowing doctors to focus on the most relevant information to their domain. Evaluation on the MIMIC-III dataset demonstrates improvements in generating domain-adapted summaries of clinical notes and hallucination reduction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Code is available at https://github.com/Lama-West/Ontology-based-decoding_EKAW2024.
- 2.
See the GitHub repository for a prompt example.
References
Abdin, M., Jacobs, S.A., et al.: Phi-3 technical report: a highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 [cs] (2024)
Adams, G., Zucker, J., Elhadad, N.: SPEER: sentence-level planning of long clinical summaries via embedded entity retrieval. arXiv preprint arXiv:2401.02369 (2024)
Adams, G., Zucker, J., Elhadad, N.: A meta-evaluation of faithfulness metrics for long-form hospital-course summarization. arXiv preprint arXiv:2303.03948 [cs] (2023)
Alsentzer, E., et al.: Publicly available clinical BERT embeddings. arXiv preprint arXiv:1904.03323 [cs] (2019)
Chern, I.C., Wang, Z., Das, S., Sharma, B., Liu, P., Neubig, G.: Improving factuality of abstractive summarization via contrastive reward learning. arXiv preprint arXiv:2307.04507 [cs] (2023)
Choi, S., Fang, T., Wang, Z., Song, Y.: KCTS: knowledge-constrained tree search decoding with token-level hallucination detection. arXiv preprint arXiv:2310.09044 [cs] (2023)
Chuang, Y.N., Tang, R., Jiang, X., Hu, X.: SPeC: a soft prompt-based calibration on performance variability of large language model in clinical notes summarization. arXiv preprint arXiv:2303.13035 [cs] (2023)
Cohen, R., Hamri, M., Geva, M., Globerson, A.: LM vs LM: detecting factual errors via cross examination. arXiv preprint arXiv:2305.13281 [cs] (2023)
Dhuliawala, S., et al.: Chain-of-verification reduces hallucination in large language models. arXiv preprint arXiv:2309.11495 (2023)
Gao, Y., et al.: Leveraging a medical knowledge graph into large language models for diagnosis prediction. arXiv:2308.14321 [cs] (2023)
Geng, S., Josifoski, M., Peyrard, M., West, R.: Grammar-constrained decoding for structured NLP tasks without finetuning. arXiv:2305.13971 [cs] (2023)
Guerreiro, N.M., et al.: Hallucinations in large multilingual translation models. arXiv:2303.16104 [cs] (2023)
He, P., Liu, X., Gao, J., Chen, W.: Deberta: decoding-enhanced bert with disentangled attention. In: International Conference on Learning Representations (2021). https://openreview.net/forum?id=XPZIaotutsD
Johnson, A.E., et al.: MIMIC-III, a freely accessible critical care database. Scientific Data 3(1), 160035 (2016). https://doi.org/10.1038/sdata.2016.35
Kraljevic, Z., et al.: Multi-domain clinical natural language processing with MedCAT: the medical concept annotation toolkit. Artif. Intell. Med. 117, 102083 (2021). https://doi.org/10.1016/j.artmed.2021.102083
Lewis, P., et al.: Retrieval-augmented generation for knowledge-intensive NLP tasks. arXiv preprint arXiv:2005.11401 (2020)
Li, K., Patel, O., Viégas, F., Pfister, H., Wattenberg, M.: Inference-time intervention: eliciting truthful answers from a language model. arXiv preprint arXiv:2306.03341 [cs] (2023)
Liu, M., Zhang, D., Tan, W., Zhang, H.: DeakinNLP at ProbSum 2023: clinical progress note summarization with rules and language ModelsClinical progress note summarization with rules and languague models. In: The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pp. 491–496. Association for Computational Linguistics, Toronto (2023).https://doi.org/10.18653/v1/2023.bionlp-1.47
Logan IV, R.L., Liu, N.F., Peters, M.E., Gardner, M., Singh, S.: Barack’s Wife Hillary: using knowledge-graphs for fact-aware language modeling. arXiv preprint arXiv:1906.07241 [cs] (2019)
Lu, X., West, P., Zellers, R., Bras, R.L., Bhagavatula, C., Choi, Y.: NeuroLogic decoding: (un)supervised neural text generation with predicate logic constraints. arXiv preprint arXiv:2010.12884 [cs] (2021)
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N.A., Lewis, M.: Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350 [cs] (2023)
Searle, T., Ibrahim, Z., Teo, J., Dobson, R.J.: Discharge summary hospital course summarisation of in patient Electronic Health Record text with clinical concept guided deep pre-trained Transformer models. J. Biomed. Inform. 141, 104358 (2023)
Shi, J., Gao, X., Kinsman, W.C., Ha, C., Gao, G.G., Chen, Y.: DI++: a deep learning system for patient condition identification in clinical notes. Artif. Intell. Med. 123, 102224 (2022)
Sorita, A., et al.: The ideal hospital discharge summary: a survey of U.S. physicians. J. Patient Safety 17(7) (2021). https://journals.lww.com/journalpatientsafety/fulltext/2021/10000/the_ideal_hospital_discharge_summary__a_survey_of.16.aspx
Stearns, M.Q., Price, C., Spackman, K.A., Wang, A.Y.: SNOMED clinical terms: overview of the development process and project status. In: Proceedings of the AMIA Symposium, pp. 662–666 (2001)
Stengel-Eskin, E., Rawlins, K., Van Durme, B.: Zero and few-shot semantic parsing with ambiguous inputs. arXiv preprint arXiv:2306.00824 [cs] (2024)
Tajirian, T., et al.: The influence of electronic health record use on physician burnout: cross-sectional survey. J. Med. Internet Res. 22(7), e19274 (2020)
Tunstall, L., et al.: Zephyr: direct distillation of LM alignment. arXiv:2310.16944 [cs] (2023)
Van Veen, D., et al.: Clinical text summarization: adapting large language models can outperform human experts. arXiv preprint arXiv:2309.07430 [cs] (2023)
Van Veen, D., et al.: Adapted large language models can outperform medical experts in clinical text summarization. arXiv preprint arXiv:2309.07430 [cs] (2024)
Vijayakumar, A.K., et al.: Diverse beam search: decoding diverse solutions from neural sequence models. arXiv preprint arXiv:1610.02424 [cs] (2018)
Wang, Y., Zhang, Z., Wang, R.: Element-aware summarization with large language models: expert-aligned evaluation and chain-of-thought method. arXiv preprint arXiv:2305.13412 [cs] (2023)
Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 (2022)
Willard, B.T., Louf, R.: Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702 [cs] (2023)
Williams, A., Nangia, N., Bowman, S.: A broad-coverage challenge corpus for sentence understanding through inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112–1122. Association for Computational Linguistics (2018). http://aclweb.org/anthology/N18-1101
Wu, Z., et al.: Fine-grained human feedback gives better rewards for language model training. arXiv preprint arXiv:2306.01693 [cs] (2023)
Yang, K., Klein, D.: FUDGE: controlled text generation with future discriminators. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3511–3535 (2021). https://doi.org/10.18653/v1/2021.naacl-main.276
van Zandvoort, D., Wiersema, L., Huibers, T., van Dulmen, S., Brinkkemper, S.: Enhancing summarization performance through transformer-based prompt engineering in automated medical reporting. arXiv preprint arXiv:2311.13274 [cs] (2024)
Zheng, L., et al.: SGLang: efficient execution of structured language model programs. arXiv preprint arXiv:2312.07104 [cs] (2024)
Acknowledgements
We gratefully acknowledge the financial support provided by the FRQS (Fonds de Recherche du Québec - Santé) Chair in Artificial Intelligence and Digital Health. This research also benefited from the computational resources and GPU infrastructure provided by the Digital Research Alliance of Canada (DRAC) and Calcul Québec. We extend our sincere thanks to Prof. John Kildea for his valuable input during initial discussions on the challenge of adapting summaries for specialists.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mehenni, G., Zouaq, A. (2025). Ontology-Constrained Generation of Domain-Specific Clinical Summaries. In: Alam, M., Rospocher, M., van Erp, M., Hollink, L., Gesese, G.A. (eds) Knowledge Engineering and Knowledge Management. EKAW 2024. Lecture Notes in Computer Science(), vol 15370. Springer, Cham. https://doi.org/10.1007/978-3-031-77792-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-77792-9_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-77791-2
Online ISBN: 978-3-031-77792-9
eBook Packages: Computer ScienceComputer Science (R0)