Download PDF
CC BY-NC-ND 4.0 · Appl Clin Inform 2019; 10(02): 295-306
DOI: 10.1055/s-0039-1688478
Research Article
Georg Thieme Verlag KG Stuttgart · New York

Diffusing an Innovation: Clinician Perceptions of Continuous Predictive Analytics Monitoring in Intensive Care

Rebecca R. Kitzmiller
1   School of Nursing, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States
,
Ashley Vaughan
1   School of Nursing, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States
,
Angela Skeeles-Worley
2   Curry School of Education and Human Development, University of Virginia, Charlottesville, Virginia, United States
,
Jessica Keim-Malpass
3   School of Nursing, University of Virginia, Charlottesville, Virginia, United States
,
Tracey L. Yap
4   School of Nursing, Duke University, Durham, North Carolina, United States
,
Curt Lindberg
5   Billings Clinic, Billings, Montana, United States
,
Susan Kennerly
6   College of Nursing, East Carolina University, Greenville, North Carolina¸ United States
,
Claire Mitchell
2   Curry School of Education and Human Development, University of Virginia, Charlottesville, Virginia, United States
,
Robert Tai
2   Curry School of Education and Human Development, University of Virginia, Charlottesville, Virginia, United States
,
Brynne A. Sullivan
7   Division of Neonatology, University of Virginia, Charlottesville, Virginia, United States
,
Ruth Anderson
1   School of Nursing, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States
,
Joseph R. Moorman
8   Departments of Cardiology and Biomedical Engineering, University of Virginia, Charlottesville, Virginia, United States
9   Center for Advanced Medical Analytics, University of Virginia, Charlottesville, Virginia, United States
› Author Affiliations Funding This study was funded by the National Center for Advancing Translational Sciences (Grant/Award Number: ‘KL2TR001109’), Mitre Corporation (Grant/Award Number: ‘Contract No 109140-Phase 1 & 2’), and University of Virginia (Grant/Award Number: ‘THRIV’).
Further Information

Publication History

29 October 2018

18 March 2019

Publication Date:
01 May 2019 (online)

Also available at
   Permissions and Reprints

Abstract

Background The purpose of this article is to describe neonatal intensive care unit clinician perceptions of a continuous predictive analytics technology and how those perceptions influenced clinician adoption. Adopting and integrating new technology into care is notoriously slow and difficult; realizing expected gains remain a challenge.

Methods Semistructured interviews from a cross-section of neonatal physicians (n = 14) and nurses (n = 8) from a single U.S. medical center were collected 18 months following the conclusion of the predictive monitoring technology randomized control trial. Following qualitative descriptive analysis, innovation attributes from Diffusion of Innovation Theory-guided thematic development.

Results Results suggest that the combination of physical location as well as lack of integration into work flow or methods of using data in care decisionmaking may have delayed clinicians from routinely paying attention to the data. Once data were routinely collected, documented, and reported during patient rounds and patient handoffs, clinicians came to view data as another vital sign. Through clinicians' observation of senior physicians and nurses, and ongoing dialogue about data trends and patient status, clinicians learned how to integrate these data in care decision making (e.g., differential diagnosis) and came to value the technology as beneficial to care delivery.

Discussion The use of newly created predictive technologies that provide early warning of illness may require implementation strategies that acknowledge the risk–benefit of treatment clinicians must balance and take advantage of existing clinician training methods.

Keywords

predictive analytics - continuous predictive monitoring - neonatal intensive care - sepsis - Diffusion of Innovations - innovation attributes - implementation

Authors' Contributions

C.L., J.R.M., R.A., R.K., and R.T. developed the study design. R.T. conducted participant interviews. A.V., C.L., R.A., R.K., S.K., and T.Y. lead the application of the theoretical framework. Data analysis was conducted by A.S.-W., A.V., C.K., C.M., J.K.-M., R.A., R.K., S.K., and T.Y. Drafts of the manuscript were prepared by A.S.-W., A.V., B.S., C.L., J.K.-M., J.R.M., R.A., R.K., S.K., and T.Y. All authors read and approved the final manuscript.


Protection of Human and Animal Subjects

Ethics approval was obtained from the University of Virginia IRB-SBS #2015–0352. Procedures included participant consent to participate.


 

  • References

  • 1 Agency for Healthcare Research and Quality. AHRQ Issue Brief: Harnessing the Power of Data. Agency for Healthcare Research and Quality; August 2015
    PubMed
  • 2 Seely A, Newman K, Herry C. Monitoring variability and complexity at the bedside. In: Sturmberg JP. , ed. The Value of Systems and Complexity Sciences for Healthcare. New York: Springer; 2016
    Search in Google Scholar
  • 3 Green GC, Bradley B, Bravi A, Seely AJE. Continuous multiorgan variability analysis to track severity of organ failure in critically ill patients. J Crit Care 2013; 28 (05) 879.e1-879.e11
    CrossrefPubMedSearch in Google Scholar
  • 4 Henry KE, Hager DN, Pronovost PJ, Saria S. A targeted real-time early warning score (TREWScore) for septic shock. Sci Transl Med 2015; 7 (299) 299ra122
    CrossrefPubMedSearch in Google Scholar
  • 5 Kappen TH, van Klei WA, van Wolfswinkel L, Kalkman CJ, Vergouwe Y, Moons KGM. Evaluating the impact of prediction models: lessons learned, challenges, and recommendations. Diagnostic and Prognostic Research 2018; 2 (01) 11
    CrossrefPubMedSearch in Google Scholar
  • 6 Kennedy G, Gallego B. Clinical prediction rules: asystematic review of healthcare provider opinions and preferences. Int J Med Inform 2019; 123: 1-10
    CrossrefPubMedSearch in Google Scholar
  • 7 Nakamura F, Nakai M. Prediction models. 　- why are they used or not used?. Circ J 2017; 81 (12) 1766-1767
    CrossrefPubMedSearch in Google Scholar
  • 8 Shelov E, Muthu N, Wolfe H. , et al. Design and implementation of a pediatric ICU acuity scoring tool as clinical decision support. Appl Clin Inform 2018; 9 (03) 576-587
    Thieme ConnectPubMedSearch in Google Scholar
  • 9 Chalmers JD, Singanayagam A, Akram AR. , et al. Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia. Systematic review and meta-analysis. Thorax 2010; 65 (10) 878-883
    CrossrefPubMedSearch in Google Scholar
  • 10 Hosein FS, Bobrovitz N, Berthelot S, Zygun D, Ghali WA, Stelfox HT. A systematic review of tools for predicting severe adverse events following patient discharge from intensive care units. Crit Care 2013; 17 (03) R102
    CrossrefPubMedSearch in Google Scholar
  • 11 Breslow MJ, Badawi O. Severity scoring in the critically ill: part 1--interpretation and accuracy of outcome prediction scoring systems. Chest 2012; 141 (01) 245-252
    CrossrefPubMedSearch in Google Scholar
  • 12 van den Boogaard M, Schoonhoven L, Maseda E. , et al. Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study. Intensive Care Med 2014; 40 (03) 361-369
    CrossrefPubMedSearch in Google Scholar
  • 13 Wassenaar A, van den Boogaard M, van Achterberg T. , et al. Multinational development and validation of an early prediction model for delirium in ICU patients. Intensive Care Med 2015; 41 (06) 1048-1056
    CrossrefPubMedSearch in Google Scholar
  • 14 Seely AJ, Bravi A, Herry C. , et al; Canadian Critical Care Trials Group (CCCTG). Do heart and respiratory rate variability improve prediction of extubation outcomes in critically ill patients?. Crit Care 2014; 18 (02) R65
    CrossrefPubMedSearch in Google Scholar
  • 15 Müller-Riemenschneider F, Holmberg C, Rieckmann N. , et al. Barriers to routine risk-score use for healthy primary care patients: survey and qualitative study. Arch Intern Med 2010; 170 (08) 719-724
    CrossrefPubMedSearch in Google Scholar
  • 16 Koppel R, Wetterneck T, Telles JL, Karsh BT. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc 2008; 15 (04) 408-423
    CrossrefPubMedSearch in Google Scholar
  • 17 Barber N, Cornford T, Klecun E. Qualitative evaluation of an electronic prescribing and administration system. Qual Saf Health Care 2007; 16 (04) 271-278
    CrossrefPubMedSearch in Google Scholar
  • 18 Bar-Lev S, Harrison MI. Negotiating time scripts during implementation of an electronic medical record. Health Care Manage Rev 2006; 31 (01) 11-17
    CrossrefPubMedSearch in Google Scholar
  • 19 Doolin B. Power and resistance in the implementation of a medical management information system. Inf Syst J 2004; 14 (04) 343-362
    CrossrefPubMedSearch in Google Scholar
  • 20 Lapointe L, Rivard S. A multilevel model of resistance to information technology implementation. Manage Inf Syst Q 2005; 29 (03) 461-491
    CrossrefPubMedSearch in Google Scholar
  • 21 Lau F, Penn A, Wilson D, Noseworthy T, Vincent D, Doze S. The diffusion of an evidence-based disease guidance system for managing stroke. Int J Med Inform 1998; 51 (2-3): 107-116
    CrossrefPubMedSearch in Google Scholar
  • 22 Littlejohns P, Wyatt JC, Garvican L. Evaluating computerised health information systems: hard lessons still to be learnt. BMJ 2003; 326 (7394): 860-863
    CrossrefPubMedSearch in Google Scholar
  • 23 Koppel R, Metlay JP, Cohen A. , et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10) 1197-1203
    CrossrefPubMedSearch in Google Scholar
  • 24 Novak L, Brooks J, Gadd C, Anders S, Lorenzi N. Mediating the intersections of organizational routines during the introduction of a health IT system. Eur J Inf Syst 2012; 21 (05) 552-569
    CrossrefPubMedSearch in Google Scholar
  • 25 Edmondson AC, Bohmer RM, Pisano GP. Disrupted routines: team learning and new technology implementation in hospitals. Adm Sci Q 2001; 46 (04) 685
    CrossrefPubMedSearch in Google Scholar
  • 26 Kappen TH, Moons KG, van Wolfswinkel L, Kalkman CJ, Vergouwe Y, van Klei WA. Impact of risk assessments on prophylactic antiemetic prescription and the incidence of postoperative nausea and vomiting: a cluster-randomized trial. Anesthesiology 2014; 120 (02) 343-354
    CrossrefPubMedSearch in Google Scholar
  • 27 Kappen TH, van Loon K, Kappen MA. , et al. Barriers and facilitators perceived by physicians when using prediction models in practice. J Clin Epidemiol 2016; 70: 136-145
    CrossrefPubMedSearch in Google Scholar
  • 28 Griffin MP, Moorman JR. Toward the early diagnosis of neonatal sepsis and sepsis-like illness using novel heart rate analysis. Pediatrics 2001; 107 (01) 97-104
    CrossrefPubMedSearch in Google Scholar
  • 29 Moorman JR, Carlo WA, Kattwinkel J. , et al. Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: a randomized trial. J Pediatr 2011; 159 (06) 900-906
    CrossrefPubMedSearch in Google Scholar
  • 30 Sullivan BA, Grice SM, Lake DE, Moorman JR, Fairchild KD. Infection and other clinical correlates of abnormal heart rate characteristics in preterm infants. J Pediatr 2014; 164 (04) 775-780
    CrossrefPubMedSearch in Google Scholar
  • 31 Goel N, Chakraborty M, Watkins WJ, Banerjee S. Predicting extubation outcomes-a model incorporating heart rate characteristics index. J Pediatr 2018; 195: 53-58
    CrossrefPubMedSearch in Google Scholar
  • 32 Clark MT, Vergales BD, Paget-Brown AO. , et al. Predictive monitoring for respiratory decompensation leading to urgent unplanned intubation in the neonatal intensive care unit. Pediatr Res 2013; 73 (01) 104-110
    CrossrefPubMedSearch in Google Scholar
  • 33 Yap TL, Kennerly S, Corazzini K, Porter K, Toles M, Anderson RA. Evaluation of Cueing Innovation for Pressure Ulcer Prevention Using Staff Focus Groups. Paper presented at: Healthcare 2014
    PubMed
  • 34 Dharampal N, Cameron C, Dixon E, Ghali W, Quan ML. Attitudes and beliefs about the surgical safety checklist: just another tick box?. Can J Surg 2016; 59 (04) 268-275
    CrossrefPubMedSearch in Google Scholar
  • 35 Federman AD, Sanchez-Munoz A, Jandorf L, Salmon C, Wolf MS, Kannry J. Patient and clinician perspectives on the outpatient after-visit summary: a qualitative study to inform improvements in visit summary design. J Am Med Inform Assoc 2017; 24 (e1): e61-e68
    CrossrefPubMedSearch in Google Scholar
  • 36 Vedel I, Akhlaghpour S, Vaghefi I, Bergman H, Lapointe L. Health information technologies in geriatrics and gerontology: a mixed systematic review. J Am Med Inform Assoc 2013; 20 (06) 1109-1119
    CrossrefPubMedSearch in Google Scholar
  • 37 Rogers E. Diffusion of Innovations. 5th ed. New York: Free Press; 2003
    Search in Google Scholar
  • 38 Simunovic M, Coates A, Smith A, Thabane L, Goldsmith CH, Levine MN. Uptake of an innovation in surgery: observations from the cluster-randomized Quality Initiative in Rectal Cancer trial. Can J Surg 2013; 56 (06) 415-421
    CrossrefPubMedSearch in Google Scholar
  • 39 Mikkelsen ME, Gaieski DF, Goyal M. , et al. Factors associated with nonadherence to early goal-directed therapy in the ED. Chest 2010; 138 (03) 551-558
    CrossrefPubMedSearch in Google Scholar
  • 40 Bunkenborg G, Poulsen I, Samuelson K, Ladelund S, Åkeson J. Mandatory early warning scoring--implementation evaluated with a mixed-methods approach. Appl Nurs Res 2016; 29: 168-176
    CrossrefPubMedSearch in Google Scholar
  • 41 Pisano GP, Bohmer RM, Edmondson AC. Organizational differences in rates of learning: evidence from the adoption of minimally invasive cardiac surgery. Manage Sci 2001; 47 (06) 752-768
    CrossrefPubMedSearch in Google Scholar
  • 42 Sandelowski M. Whatever happened to qualitative description?. Res Nurs Health 2000; 23 (04) 334-340
    CrossrefPubMedSearch in Google Scholar
  • 43 Hinds PS, Vogel RJ, Clarke-Steffen L. The possibilities and pitfalls of doing a secondary analysis of a qualitative data set. Qual Health Res 1997; 7 (03) 408-424
    CrossrefPubMedSearch in Google Scholar
  • 44 Szabo V, Strang VR. Secondary analysis of qualitative data. ANS Adv Nurs Sci 1997; 20 (02) 66-74
    PubMedSearch in Google Scholar
  • 45 Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005; 15 (09) 1277-1288
    CrossrefPubMedSearch in Google Scholar
  • 46 Miles MB, Huberman AM, Saldaña J. Qualitative Data Analysis: A Methods Sourcebook. Thousand Oaks, CA: SAGE Publications, Incorporated; 2013
    Search in Google Scholar
  • 47 Griffin MP, Lake DE, Bissonette EA, Harrell Jr FE, O'Shea TM, Moorman JR. Heart rate characteristics: novel physiomarkers to predict neonatal infection and death. Pediatrics 2005; 116 (05) 1070-1074
    CrossrefPubMedSearch in Google Scholar
  • 48 Ford EW, McAlearney AS, Phillips MT, Menachemi N, Rudolph B. Predicting computerized physician order entry system adoption in US hospitals: can the federal mandate be met?. Int J Med Inform 2008; 77 (08) 539-545
    CrossrefPubMedSearch in Google Scholar
  • 49 Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling N. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study. Ann R Coll Surg Engl 2006; 88 (06) 571-575
    CrossrefPubMedSearch in Google Scholar
  • 50 Priestley G, Watson W, Rashidian A. , et al. Introducing critical care outreach: a ward-randomised trial of phased introduction in a general hospital. Intensive Care Med 2004; 30 (07) 1398-1404
    PubMedSearch in Google Scholar
  • 51 Guilbert ER, Robitaille J, Guilbert AC, Morin D. Determinants of the implementation of a new practice in hormonal contraception by Quebec nurses. Can J Hum Sex 2014; 23 (01) 34-48
    PubMedSearch in Google Scholar
  • 52 Weick KE, Sutcliffe KM, Obstfeld D. Organizing and the process of sensemaking. Organ Sci 2005; 16 (04) 409
    CrossrefPubMedSearch in Google Scholar
  • 53 Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82 (04) 581-629
    CrossrefPubMedSearch in Google Scholar
  • 54 Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit. Appl Clin Inform 2018; 9 (02) 268-274
    Thieme ConnectPubMedSearch in Google Scholar
  • 55 Moss TJ, Lake DE, Calland JF. , et al. Signatures of subacute potentially catastrophic illness in the ICU: model development and validation. Crit Care Med 2016; 44 (09) 1639-1648
    CrossrefPubMedSearch in Google Scholar
  • 56 Weick KE. Sensemaking in Organizations. London: Sage; 1995
    Search in Google Scholar
  • 57 Norman G. Research in clinical reasoning: past history and current trends. Med Educ 2005; 39 (04) 418-427
    CrossrefPubMedSearch in Google Scholar
  • 58 Cruess SR, Cruess RL, Steinert Y. Role modelling--making the most of a powerful teaching strategy. BMJ 2008; 336 (7646): 718-721
    CrossrefPubMedSearch in Google Scholar
  • 59 Masterson Creber RM, Dayan PS, Kuppermann N. , et al; Pediatric Emergency Care Applied Research Network (PECARN) and the Clinical Research on Emergency Services and Treatments (CREST) Network. Applying the RE-AIM framework for the evaluation of a clinical decision support tool for pediatric head trauma: a mixed-methods study. Appl Clin Inform 2018; 9 (03) 693-703
    Thieme ConnectPubMedSearch in Google Scholar
  • 60 Menachemi N, Burke DE, Ayers DJ. Factors affecting the adoption of telemedicine--a multiple adopter perspective. J Med Syst 2004; 28 (06) 617-632
    CrossrefPubMedSearch in Google Scholar
  • 61 Fitzgerald L, Ferlie E, Wood M, Hawkins C. Interlocking interactions, the diffusion of innovations in health care. Hum Relat 2002; 55 (12) 1429
    CrossrefPubMedSearch in Google Scholar
  • 62 Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM 2001; 94 (10) 521-526
    CrossrefPubMedSearch in Google Scholar
  • 63 Heitz CR, Gaillard JP, Blumstein H, Case D, Messick C, Miller CD. Performance of the maximum modified early warning score to predict the need for higher care utilization among admitted emergency department patients. J Hosp Med 2010; 5 (01) E46-E52
    CrossrefPubMedSearch in Google Scholar
  • 64 Keim-Malpass J, Kitzmiller RR, Skeeles-Worley A. , et al. Advancing continuous predictive analytics monitoring: Moving from implementation to clinical action in a learning health system. Crit Care Nurs Clin North Am 2018; 30 (02) 273-287
    CrossrefPubMedSearch in Google Scholar