{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T15:19:52Z","timestamp":1725031192150},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,5,4]],"date-time":"2023-05-04T00:00:00Z","timestamp":1683158400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,4]],"date-time":"2023-05-04T00:00:00Z","timestamp":1683158400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Department of Critical Care Medicine, Quanzhou First Hospital Affiliated to Fujian Medical University"},{"name":"Shanghai Nuanhe Brain Technology Co., Ltd, China"},{"name":"Shanghai Biotecan Pharmaceuticals Co., Ltd, China"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"Abstract<\/jats:title>\n Background<\/jats:title>\n A growing body of research suggests that the use of computerized decision support systems can better guide disease treatment and reduce the use of social and medical resources. Artificial intelligence (AI) technology is increasingly being used in medical decision-making systems to obtain optimal dosing combinations and improve the survival rate of sepsis patients. To meet the real-world requirements of medical applications and make the training model more robust, we replaced the core algorithm applied in an AI-based medical decision support system developed by research teams at the Massachusetts Institute of Technology (MIT) and IMPERIAL College London (ICL) with the deep deterministic policy gradient (DDPG) algorithm. The main objective of this study was to develop an AI-based medical decision-making system that makes decisions closer to those of professional human clinicians and effectively reduces the mortality rate of sepsis patients.<\/jats:p>\n <\/jats:sec>\n Methods<\/jats:title>\n We used the same public intensive care unit (ICU) dataset applied by the research teams at MIT and ICL, i.e., the Multiparameter Intelligent Monitoring in Intensive Care III (MIMIC-III) dataset, which contains information on the hospitalizations of 38,600 adult sepsis patients over the age of 15. We applied the DDPG algorithm as a strategy-based reinforcement learning approach to construct an AI-based medical decision-making system and analyzed the model results within a two-dimensional space to obtain the optimal dosing combination decision for sepsis patients.<\/jats:p>\n <\/jats:sec>\n Results<\/jats:title>\n The results show that when the clinician administered the exact same dose as that recommended by the AI model, the mortality of the patients reached the lowest rate at 11.59%. At the same time, according to the database, the baseline mortality rate of the patients was calculated as 15.7%. This indicates that the patient mortality rate when difference between the doses administered by clinicians and those determined by the AI model was zero was approximately 4.2% lower than the baseline patient mortality rate found in the dataset. The results also illustrate that when a clinician administered a different dose than that recommended by the AI model, the patient mortality rate increased, and the greater the difference in dose, the higher the patient mortality rate. Furthermore, compared with the medical decision-making system based on the Deep-Q Learning Network (DQN) algorithm developed by the research teams at MIT and ICL, the optimal dosing combination recommended by our model is closer to that given by professional clinicians. Specifically, the number of patient samples administered by clinicians with the exact same dose recommended by our AI model increased by 142.3% compared with the model based on the DQN algorithm, with a reduction in the patient mortality rate of 2.58%.<\/jats:p>\n <\/jats:sec>\n Conclusions<\/jats:title>\n The treatment plan generated by our medical decision-making system based on the DDPG algorithm is closer to that of a professional human clinician with a lower mortality rate in hospitalized sepsis patients, which can better help human clinicians deal with complex conditional changes in sepsis patients in an ICU. Our proposed AI-based medical decision-making system has the potential to provide the best reference dosing combinations for additional drugs.<\/jats:p>\n <\/jats:sec>","DOI":"10.1186\/s12911-023-02175-7","type":"journal-article","created":{"date-parts":[[2023,5,4]],"date-time":"2023-05-04T14:03:45Z","timestamp":1683209025000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients"],"prefix":"10.1186","volume":"23","author":[{"given":"Tianlai","family":"Lin","sequence":"first","affiliation":[]},{"given":"Xinjue","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Jianbing","family":"Gong","sequence":"additional","affiliation":[]},{"given":"Rundong","family":"Tan","sequence":"additional","affiliation":[]},{"given":"Weiming","family":"Li","sequence":"additional","affiliation":[]},{"given":"Lijun","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Yingxia","family":"Pan","sequence":"additional","affiliation":[]},{"given":"Xiang","family":"Xu","sequence":"additional","affiliation":[]},{"given":"Junhui","family":"Gao","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,5,4]]},"reference":[{"issue":"5","key":"2175_CR1","first-page":"581614","volume":"15","author":"J Cohen","year":"2006","unstructured":"Cohen J, Vincent J-L, Adhikari NKJ, Machado FR, Angus DC, Calandra T, Jaton K, Giulieri S, Delaloye J, Opal S, Tracey K, van der Poll T, Pelfrene E. Sepsis: a roadmap for future research. Lancet Infectious Diseases. 2006;15(5):581614.","journal-title":"Lancet Infectious Diseases"},{"issue":"8","key":"2175_CR2","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1001\/jama.2016.0287","volume":"315","author":"M Singer","year":"2016","unstructured":"Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801\u201310.","journal-title":"JAMA"},{"issue":"10219","key":"2175_CR3","doi-asserted-by":"publisher","first-page":"200","DOI":"10.1016\/S0140-6736(19)32989-7","volume":"395","author":"KE Rudd","year":"2020","unstructured":"Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, Colombara DV, et al. Global, regional, and national sepsis incidence and mortality, 1990\u20132017: analysis for the Global Burden of Disease Study. Lancet. 2020;395(10219):200\u201311.","journal-title":"Lancet"},{"key":"2175_CR4","doi-asserted-by":"publisher","first-page":"16045","DOI":"10.1038\/nrdp.2016.45","volume":"2","author":"RS Hotchkiss","year":"2016","unstructured":"Hotchkiss RS, Moldawer LL, Opal SM, Reinhart K, Turnbull IR, Vincent J-L. Sepsis and septic shock. Nat Rev Dis Primers. 2016;2:16045.","journal-title":"Nat Rev Dis Primers"},{"issue":"3","key":"2175_CR5","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1007\/s15010-008-8203-z","volume":"37","author":"R Beale","year":"2009","unstructured":"Beale R, Reinhart K, Brunkhorst FM, et al. Promoting Global Research Excellence in Severe Sepsis (PROGRESS): Lessons from an International Sepsis Registry. Infection. 2009;37(3):222\u201332.","journal-title":"Infection"},{"issue":"12","key":"2175_CR6","doi-asserted-by":"publisher","first-page":"1889","DOI":"10.1097\/CCM.0000000000003342","volume":"46","author":"CJ Paoli","year":"2018","unstructured":"Paoli CJ, Reynolds MA, Sinha M, Gitlin M, Crouse E. Epidemiology and costs of sepsis in the United States\u2014an analysis based on timing of diagnosis and severity level. Observational Study. Crit Care Med. 2018;46(12):1889\u201397.","journal-title":"Crit Care Med"},{"issue":"10","key":"2175_CR7","doi-asserted-by":"publisher","first-page":"2158","DOI":"10.1097\/CCM.0000000000000520","volume":"42","author":"J Waechter","year":"2014","unstructured":"Waechter J, Kumar A, Lapinsky SE, Marshall J, Dodek P, Arabi Y, Parrillo JE, Dellinger RP, Garland A, Cooperative Antimicrobial Therapy of Septic Shock Database Research Group, et al. Interaction between fluids and vasoactive agents on mortality in septic shock: a multicenter, observational study. Crit Care Med. 2014;42(10):2158\u201368.","journal-title":"Crit Care Med"},{"issue":"3","key":"2175_CR8","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1007\/s00134-017-4683-6","volume":"43","author":"A Rhodes","year":"2017","unstructured":"Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, Kumar A, Sevransky JE, Sprung CL, Nunnally ME, et al. Surviving sepsis campaign: international guidelines for the management of sepsis and septic shock: 2016. Intensive Care Med. 2017;43(3):304\u201377.","journal-title":"Intensive Care Med"},{"issue":"5","key":"2175_CR9","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1111\/aas.12479","volume":"59","author":"PE Marik","year":"2015","unstructured":"Marik PE. The demise of early goal-directed therapy for severe sepsis and septic shock. Acta Anaesthesiol Scand. 2015;59(5):561\u20137.","journal-title":"Acta Anaesthesiol Scand"},{"key":"2175_CR10","volume-title":"Dueling network architectures for deep reinforcement learning","author":"Z Wang","year":"2015","unstructured":"Wang Z, de Freitas N, Lanctot M. Dueling network architectures for deep reinforcement learning. 2015. CoRR, abs\/1511.06581."},{"key":"2175_CR11","doi-asserted-by":"publisher","unstructured":"van Hasselt H, Guez A, Silver D. Deep Reinforcement learning with double Q-learning. Proceedings of the AAAI conference on artificial intelligence. 2016;30(1).\u00a0 https:\/\/doi.org\/10.1609\/aaai.v30i1.10295.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"2175_CR12","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D. Human-level control through deep reinforcement learning. Nature. 2015;518:529\u201333.","journal-title":"Nature"},{"key":"2175_CR13","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D. Mastering the game of Go with deep neural networks and tree search. Nature. 2016;529:484\u20139.","journal-title":"Nature"},{"issue":"3","key":"2175_CR14","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/s10728-021-00430-4","volume":"29","author":"S Holm","year":"2021","unstructured":"Holm S, Stanton C, Bartlett B. A new argument for no-fault compensation in health care: the introduction of artificial intelligence systems. Health Care Anal. 2021;29(3):171\u201388. https:\/\/doi.org\/10.1007\/s10728-021-00430-4. Epub 2021 Mar 21. PMID: 33745121; PMCID: PMC8321978.","journal-title":"Health Care Anal."},{"issue":"10","key":"2175_CR15","doi-asserted-by":"publisher","first-page":"2646","DOI":"10.21037\/tp-20-310","volume":"10","author":"S Ranjit","year":"2021","unstructured":"Ranjit S, Kissoon N. Challenges and Solutions in translating sepsis guidelines into practice in resource-limited settings. Transl Pediatr. 2021;10(10):2646\u201365. https:\/\/doi.org\/10.21037\/tp-20-310.","journal-title":"Transl Pediatr"},{"key":"2175_CR16","doi-asserted-by":"publisher","first-page":"739728","DOI":"10.3389\/fimmu.2021.739728","volume":"12","author":"JA Balch","year":"2021","unstructured":"Balch JA, Delitto D, Tighe PJ, et al. Machine learning applications in solid organ transplantation and related complications. Front Immunol. 2021;12:739728 Published 2021 Sep 16.\nhttps:\/\/doi.org\/10.3389\/fimmu.2021.739728.","journal-title":"Front Immunol."},{"issue":"6","key":"2175_CR17","first-page":"A187","volume":"8","author":"TP Lillicrap","year":"2015","unstructured":"Lillicrap TP, Hunt JJ, Pritzel A, et al. Continuous control with deep reinforcement learning. Computer Science. 2015;8(6):A187.","journal-title":"Computer Science"},{"key":"2175_CR18","doi-asserted-by":"publisher","first-page":"2402","DOI":"10.1001\/jama.2016.17216","volume":"316","author":"V Gulshan","year":"2016","unstructured":"Gulshan V, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. J Am Med Assoc. 2016;316:2402\u201310.","journal-title":"J Am Med Assoc"},{"key":"2175_CR19","volume-title":"A reinforcement learning approach to weaning of mechanical ventilation in intensive care units","author":"N Prasad","year":"2017","unstructured":"Prasad N, Cheng LF, Chivers C, Draugelis M, Engelhardt BE. A reinforcement learning approach to weaning of mechanical ventilation in intensive care units. 2017.\u00a0Preprint at https:\/\/arxiv.org\/abs\/1704.06300."},{"key":"2175_CR20","doi-asserted-by":"publisher","first-page":"661","DOI":"10.1586\/17434440.2013.827515","volume":"10","author":"MK Bothe","year":"2013","unstructured":"Bothe MK, et al. The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas. Expert Rev Med Devices. 2013;10:661\u201373.","journal-title":"Expert Rev Med Devices"},{"key":"2175_CR21","first-page":"1414","volume-title":"Towards efficient, personalized anesthesia using continuous reinforcement learning for propofol infusion control. in International IEEE\/EMBS Conference on Neural Engineering","author":"C Lowery","year":"2013","unstructured":"Lowery C, Faisal AA. Towards efficient, personalized anesthesia using continuous reinforcement learning for propofol infusion control. in International IEEE\/EMBS Conference on Neural Engineering. San Diego, CA, USA: IEEE; 2013. p. 1414\u20137."},{"key":"2175_CR22","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"1998","unstructured":"Sutton RS, Barto AG. Reinforcement Learning: An Introduction. 1st ed. Cambridge, MA, USA: MIT Press; 1998.","edition":"1"},{"key":"2175_CR23","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.artmed.2012.12.003","volume":"57","author":"CC Bennett","year":"2013","unstructured":"Bennett CC, Hauser K. Artificial intelligence framework for simulating clinical decision-making: a Markov decision process approach. Artif Intell Med. 2013;57:9\u201319.","journal-title":"Artif Intell Med"},{"key":"2175_CR24","doi-asserted-by":"publisher","first-page":"593","DOI":"10.1007\/1-4020-8066-2_23","volume-title":"in Operations Research and Health Care","author":"AJ Schaefer","year":"2005","unstructured":"Schaefer AJ, Bailey MD, Shechter SM, Roberts MS. Modeling Medical Treatment Using Markov Decision Processes. In: Brandeau ML, Sainfort F, Pierskalla WP, editors. In Operations Research and Health Care. Boston: Springer; 2005. p. 593\u2013612."},{"issue":"1","key":"2175_CR25","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1186\/s13054-015-0970-1","volume":"19","author":"A Acheampong","year":"2015","unstructured":"Acheampong A, Vincent JL. A positive fluid balance is an independent prognostic factor in patients with sepsis. Crit Care. 2015;19(1):251.","journal-title":"Crit Care"},{"key":"2175_CR26","doi-asserted-by":"publisher","unstructured":"Johnson A, Pollard T, Shen L, et al. MIMIC-III, a freely accessible critical care database. Sci Data 3. 160035(2016). https:\/\/doi.org\/10.1038\/sdata.2016.35.","DOI":"10.1038\/sdata.2016.35"},{"key":"2175_CR27","volume-title":"In Neural Information Processing Systems Workshop on Machine Learning for Health","author":"M Komorowski","year":"2016","unstructured":"Komorowski M, Gordon A, Celi LA, Faisal A. A Markov Decision Process to suggest optimal treatment of severe infections in intensive care. In: In Neural Information Processing Systems Workshop on Machine Learning for Health. 2016."},{"key":"2175_CR28","unstructured":"Raghu A, Komorowski M, Ahmed I, et al. Deep reinforcement learning for sepsis treatment. arXiv preprint arXiv:1711.09602, 2017."},{"key":"2175_CR29","unstructured":"Gottesman O, Johansson F, Meier J, et al. Evaluating reinforcement learning algorithms in observational health settings. arXiv preprint arXiv:1805.12298, 2018."},{"key":"2175_CR30","doi-asserted-by":"publisher","first-page":"1716","DOI":"10.1038\/s41591-018-0213-5","volume":"24","author":"M Komorowski","year":"2018","unstructured":"Komorowski M, Celi LA, Badawi O, et al. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. 2018;24:1716\u201320.","journal-title":"Nat Med"},{"key":"2175_CR31","doi-asserted-by":"publisher","unstructured":"Cosgriff CV, Celi LA, Stone DJ. Critical care, critical data.\u00a0Biomed Eng Comput Biol. 2019;10. https:\/\/doi.org\/10.1177\/1179597219856564.","DOI":"10.1177\/1179597219856564"},{"key":"2175_CR32","doi-asserted-by":"publisher","unstructured":"Johnson A, Pollard T, Mark R. MIMIC-III Clinical Database (version 1.4). PhysioNet. 2016.\u00a0https:\/\/doi.org\/10.13026\/C2XW26.","DOI":"10.13026\/C2XW26"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02175-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-023-02175-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02175-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,4]],"date-time":"2023-05-04T14:04:14Z","timestamp":1683209054000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-023-02175-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,4]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2175"],"URL":"https:\/\/doi.org\/10.1186\/s12911-023-02175-7","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-1391838\/v1","asserted-by":"object"}]},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,4]]},"assertion":[{"value":"24 February 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 April 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 May 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All methods were carried out in accordance with relevant guidelines and regulations.All experimental protocols were approved based on the ethical standards of the Clinical Research Ethics Committee of the Department of Critical Care Medicine, Quanzhou First Hospital affiliated with Fujian Medical University, China. The approve number is No.188 [2020].The dataset supporting the conclusions of this article is the Medical Information Mart for Intensive Care version III (MIMIC-III) version 1.4 [CitationRef removed]. The databases are publicly deidentified; thus, informed consent and approval from the Institutional Review Board were waived. Our access to the database was approved after completion of the collaborative institutional training initiative (CITI program) web-based training course, \u201cData or Specimens Only research\u201d (Record ID:31529575).More details are available at ExternalRef removed.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"81"}}