Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis

doi:10.1080/09273948.2023.2253471

. 2024 Oct;32(8):1526-1531.

doi: 10.1080/09273948.2023.2253471. Epub 2023 Sep 18.

Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis

William Rojas-Carabali^{1

2}, Carlos Cifuentes-González³, Xin Wei¹, Ikhwanuliman Putera^{4

5

6

7}, Alok Sen⁸, Zheng Xian Thng¹, Rajdeep Agrawal², Tobias Elze⁹, Lucia Sobrin⁹, John H Kempen^{9

10

11

12}, Bernett Lee¹, Jyotirmay Biswas¹³, Quan Dong Nguyen¹⁴, Vishali Gupta¹⁵, Alejandra de-la-Torre¹, Rupesh Agrawal^{12

16

17

18}

Affiliations

¹ National Healthcare Group Eye Institute, Tan Tock Seng Hospital, Singapore, Singapore.
² Department of Bioinformatics, Lee Kong Chiang School of Medicine, Nanyang Technological University, Singapore, Singapore.
³ Neuroscience Research Group (NEUROS), Neurovitae Center for Neuroscience, Institute of Translational Medicine (IMT), Escuela de Medicina y Ciencias de la Salud, Universidad del Rosario, Bogotá, Colombia.
⁴ Department of Ophthalmology, Faculty of Medicine Universitas Indonesia - CiptoMangunkusmoKirana Eye Hospital, Jakarta, Indonesia.
⁵ Laboratory Medical Immunology, Department of Immunology, ErasmusMC, University Medical Centre, Rotterdam, the Netherlands.
⁶ Department of Internal Medicine, Division of Clinical Immunology, Erasmus MC, University Medical Center, Rotterdam, The Netherlands.
⁷ Department of Ophthalmology, Erasmus MC, University Medical Center, Rotterdam, The Netherlands.
⁸ Department of Vitreoretina and Uveitis, Sadguru Netra Chikatsalya, Chitrakoot, India.
⁹ Department of Ophthalmology, Massachusetts Eye and Ear/Harvard Medical School, and Schepens Eye Research Institute, Boston, Massachusetts, USA.
¹⁰ Community Ophthalmology, Sight for Souls, Bellevue, Washington, USA.
¹¹ Department of Ophthalmology, Addis Ababa University, Addis Ababa, Ethiopia.
¹² MyungSung Christian Medical Center (MCM) Eye Unit, MCM Comprehensive Specialized Hospital, and MyungSung Medical School, Addis Ababa, Ethiopia.
¹³ Department of Ocular Pathology and Uveitis, Medical Research Foundation, Sankara Netralaya, Chennai, India.
¹⁴ Byers Eye Institute, Stanford University, Palo Alto, California, USA.
¹⁵ Post Graduate Institute of Medical Education and Research (PGIMER), Advance Eye Centre, Chandigarh, India.
¹⁶ Department of Ophthalmology and Visual Sciences, Academic Clinical Program, Duke-NUS Medical School, Singapore, Singapore.
¹⁷ Moorfields Eye Hospital, NHS Foundation Trust, London, UK.
¹⁸ Singapore Eye Research Institute, The Academia, Singapore, Singapore.

PMID: 37722842
DOI: 10.1080/09273948.2023.2253471

Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis

William Rojas-Carabali et al. Ocul Immunol Inflamm. 2024 Oct.

. 2024 Oct;32(8):1526-1531.

doi: 10.1080/09273948.2023.2253471. Epub 2023 Sep 18.

Authors

Affiliations

¹ National Healthcare Group Eye Institute, Tan Tock Seng Hospital, Singapore, Singapore.
² Department of Bioinformatics, Lee Kong Chiang School of Medicine, Nanyang Technological University, Singapore, Singapore.
³ Neuroscience Research Group (NEUROS), Neurovitae Center for Neuroscience, Institute of Translational Medicine (IMT), Escuela de Medicina y Ciencias de la Salud, Universidad del Rosario, Bogotá, Colombia.
⁴ Department of Ophthalmology, Faculty of Medicine Universitas Indonesia - CiptoMangunkusmoKirana Eye Hospital, Jakarta, Indonesia.
⁵ Laboratory Medical Immunology, Department of Immunology, ErasmusMC, University Medical Centre, Rotterdam, the Netherlands.
⁶ Department of Internal Medicine, Division of Clinical Immunology, Erasmus MC, University Medical Center, Rotterdam, The Netherlands.
⁷ Department of Ophthalmology, Erasmus MC, University Medical Center, Rotterdam, The Netherlands.
⁸ Department of Vitreoretina and Uveitis, Sadguru Netra Chikatsalya, Chitrakoot, India.
⁹ Department of Ophthalmology, Massachusetts Eye and Ear/Harvard Medical School, and Schepens Eye Research Institute, Boston, Massachusetts, USA.
¹⁰ Community Ophthalmology, Sight for Souls, Bellevue, Washington, USA.
¹¹ Department of Ophthalmology, Addis Ababa University, Addis Ababa, Ethiopia.
¹² MyungSung Christian Medical Center (MCM) Eye Unit, MCM Comprehensive Specialized Hospital, and MyungSung Medical School, Addis Ababa, Ethiopia.
¹³ Department of Ocular Pathology and Uveitis, Medical Research Foundation, Sankara Netralaya, Chennai, India.
¹⁴ Byers Eye Institute, Stanford University, Palo Alto, California, USA.
¹⁵ Post Graduate Institute of Medical Education and Research (PGIMER), Advance Eye Centre, Chandigarh, India.
¹⁶ Department of Ophthalmology and Visual Sciences, Academic Clinical Program, Duke-NUS Medical School, Singapore, Singapore.
¹⁷ Moorfields Eye Hospital, NHS Foundation Trust, London, UK.
¹⁸ Singapore Eye Research Institute, The Academia, Singapore, Singapore.

PMID: 37722842
DOI: 10.1080/09273948.2023.2253471

Abstract

Introduction: Accurate diagnosis and timely management are vital for favorable uveitis outcomes. Artificial Intelligence (AI) holds promise in medical decision-making, particularly in ophthalmology. Yet, the diagnostic precision and management advice from AI-based uveitis chatbots lack assessment.

Methods: We appraised diagnostic accuracy and management suggestions of an AI-based chatbot, ChatGPT, versus five uveitis-trained ophthalmologists, using 25 standard cases aligned with new Uveitis Nomenclature guidelines. Participants predicted likely diagnoses, two differentials, and next management steps. Comparative success rates were computed.

Results: Ophthalmologists excelled (60-92%) in likely diagnosis, exceeding AI (60%). Considering fully and partially accurate diagnoses, ophthalmologists achieved 76-100% success; AI attained 72%. Despite an 8% AI improvement, its overall performance lagged. Ophthalmologists and AI agreed on diagnosis in 48% cases, with 91.6% exhibiting concurrence in management plans.

Conclusions: The study underscores AI chatbots' potential in uveitis diagnosis and management, indicating their value in reducing diagnostic errors. Further research is essential to enhance AI chatbot precision in diagnosis and recommendations.

Keywords: Artificial intelligence; chatbot; diagnosis; natural language processing; ophthalmology.

PubMed Disclaimer

Cited by

Applications of ChatGPT in the diagnosis, management, education, and research of retinal diseases: a scoping review.
Bellanda VCF, Santos MLD, Ferraz DA, Jorge R, Melo GB. Bellanda VCF, et al. Int J Retina Vitreous. 2024 Oct 17;10(1):79. doi: 10.1186/s40942-024-00595-9. Int J Retina Vitreous. 2024. PMID: 39420407 Free PMC article. Review.
Applications of artificial intelligence-enabled robots and chatbots in ophthalmology: recent advances and future trends.
Madadi Y, Delsoz M, Khouri AS, Boland M, Grzybowski A, Yousefi S. Madadi Y, et al. Curr Opin Ophthalmol. 2024 May 1;35(3):238-243. doi: 10.1097/ICU.0000000000001035. Epub 2024 Jan 22. Curr Opin Ophthalmol. 2024. PMID: 38277274 Review.
Utilizing Large Language Models in Ophthalmology: The Current Landscape and Challenges.
Chotcomwongse P, Ruamviboonsuk P, Grzybowski A. Chotcomwongse P, et al. Ophthalmol Ther. 2024 Oct;13(10):2543-2558. doi: 10.1007/s40123-024-01018-6. Epub 2024 Aug 24. Ophthalmol Ther. 2024. PMID: 39180701 Free PMC article. Review.
Assessment of a Large Language Model's Responses to Questions and Cases About Glaucoma and Retina Management.
Huang AS, Hirabayashi K, Barna L, Parikh D, Pasquale LR. Huang AS, et al. JAMA Ophthalmol. 2024 Apr 1;142(4):371-375. doi: 10.1001/jamaophthalmol.2023.6917. JAMA Ophthalmol. 2024. PMID: 38386351 Free PMC article.
Artificial Versus Human Intelligence in the Diagnostic Approach of Ophthalmic Case Scenarios: A Qualitative Evaluation of Performance and Consistency.
Mandalos A, Tsouris D. Mandalos A, et al. Cureus. 2024 Jun 16;16(6):e62471. doi: 10.7759/cureus.62471. eCollection 2024 Jun. Cureus. 2024. PMID: 39015855 Free PMC article.

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
- Atypon
- Taylor & Francis
Other Literature Sources
- figshare - Data

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis

Affiliations

Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis

Authors

Affiliations

Abstract

Similar articles

Cited by

MeSH terms

LinkOut - more resources

Full Text Sources

Other Literature Sources

Abstract

Similar articles

Cited by

MeSH terms

Related information

LinkOut - more resources

Full Text Sources

Other Literature Sources