HOMERuN Hospital Medicine Collaborative Research Group

Research Collaboration Proposals

Use of a modified early warning score to predict in-hospital mortality and alert clinicians to potential decompensation

Proposal Status: 

Your name and affiliations, and potential co-investigators at your site or elsewhere.

University of Utah team: Principal Investigators : Kencee Graves, MD – Hospitalist, Assistant Professor, Division of General Internal Medicine, University of Utah and Devin Horton, MD – Hospitalist, Assistant Professor, Division of General Internal Medicine, University of Utah

  1. Co-investigators:
    1. Kensaku Kawamoto, MD, PhD, MHS - Assistant Professor, Associate Chief Medical Information Officer; Director, Knowledge Management and Mobilization, Department of Biomedical Informatics
    2. Polina Kukhareva, MPH, MHS - Pre-doctoral fellow and biostatistician, Department of Biomedical Informatics
    3. Reed Barney, Lead Principal Data Warehouse Architect, University of Utah Health, Enterprise Data Warehouse
    4. Michael White, MD, MBA - Data Warehouse Architect, 4University of Utah Health, Enterprise Data Warehouse
  2. Potential co-investigators at other sites: We are pursuing collaboration with our PCORI CDRN (PaTH; http://www.pathnetwork.org), which includes: University of Pittsburgh, Geisinger Health System, Johns Hopkins University, Pennsylvania State University, and Temple University. We have presented the protocol outlined below to the PaTH Future Research Topics (FRT) Committee and have received feedback on technical feasibility and names of potential collaborators at other PaTH sites. We would also welcome collaboration from HOMERuN sites/members.

 Key clinical questions or evidence gaps you want to study (PCORI Decisional dilemma and gap analysis)

Sepsis is a leading cause of inpatient mortality and is the most expensive cause of hospitalization. Abnormal vital signs in patients with sepsis can be predictive of in-hospital mortality, as we have demonstrated with preliminary work at the University of Utah. Decompensating non-ICU patients with sepsis represent a significant safety problem and an opportunity to improve care and outcomes. Our project addresses the following clinical questions or evidence gaps:

Phase 1: Does applying the University of Utah’s Modified Early Warning Score (MEWS) to patients admitted with sepsis at other academic medical centers predict inpatient mortality (sepsis- and non-sepsis related)? What are the barriers and facilitators to potentially implementing MEWS in other academic medical centers?

Phase 2: Does implementation of a MEWS-based clinical decision support tool improve sepsis treatment in non-ICU patients admitted with sepsis and thereby reduce direct costs, ICU length of stay, sepsis-related organ injury, hospital length of stay and mortality?

Aims and Hypotheses  

Phase 1 Aim 1: Apply the Utah MEWS to 1 year of vital sign data for inpatients to evaluate the mortality rate (sepsis- and non-sepsis-related) for each MEWS score.

Phase 1 Aim 1 Hypothesis: A patient’s highest MEWS will be predictive for sepsis and non-sepsis inpatient mortality. 

Phase 1 Aim 2: Determine the barriers and facilitators to implementing a MEWS-based clinical decision support tool in a diverse group of academic medical centers.

Phase 1 Aim 2 Hypothesis: We will identify provider- and system-level barriers and facilitators to implementing a MEWS-based clinical decision support tool that will inform Phase 2 of the proposed study.

Phase 2 Aim 1: Implement the Utah MEWS clinical decision support tool at other academic medical centers.

Phase 2 Hypothesis:  A MEWS clinical decision support tool in the electronic medical record will detect septic and decompensating patients earlier than standard care, leading to more frequent surveillance of sick patients, earlier resuscitation and earlier transfer to higher levels of care.

Phase 2 Aim 2: Determine the impact of a MEWS-based clinical decision support tool on key clinical outcomes.

Phase 2 Aim 2 Hypothesis: The earlier intervention aided by clinical decision support tool  will lead to less severe organ damage and shock, decreased ICU and hospital length of stay, decreased direct costs, and decreased mortality.

Preliminary data:

Phase 1: For more information, please see attached slides. Briefly, we use an aggregated weighted vital sign system at the University of Utah that shows an association with inpatient mortality rates. The first slide shows Validated Early Warning Score (ViEWS) as published in Resuscitation 2010. ViEWS is another weighted aggregate vital sign scoring system shown to have AUROC of 0.88 for predicting mortality 24 hours after vital sign observation at one hospital in the United Kingdom. The second slide is the MEWS calculation table used at the University of Utah. The third slide shows the percentage (red) of patients with a sepsis diagnosis who died at the University of Utah over a 1 year period of time at each MEWS score (score listed was the patient’s maximum MEWS during admission). We found that septic patients with a MEWS of 5 have a 10% mortality and that mortality climbs with increasing max MEWS scores.

Phase 2: We published pilot data showing the effect of our MEWS system (Lee et al. JAMA. 2016 Sep 13;316(10):1061-1072.) This data shows that implementation of a MEWS system for patient with sepsis is associated with decreased time to antibiotics, length of stay and total direct cost. In addition, we have completed the full study and it confirms our pilot data. It is currently under review for publication. 

For further details on the JAMA article cited above, see (also attached):

http://jamanetwork.com/journals/jama/fullarticle/2552208  (table 5 for details)

Briefly, after 4 months of implementation of the MEWS system on the acute internal medicine service, the time from meeting SIRS criteria to administration of anti-infective agents had an absolute change of −4.1 hours (95% CI, −9.9 to −1.0 hours; P = .02), absolute change in LOS of -1.6 days (-5.7-0.6), and relative cost reduction of –49 (-64 to –23) with no significant increase in the use of broad spectrum anti-infective agents. 

Study design:

Phase 1 (retrospective and preparatory for implementation phase):  Apply the MEWS calculation to one year of vital sign data on admitted patients with a sepsis diagnosis at each participating institution. Collect data on inpatient mortality, total number of patients admitted with sepsis diagnosis, and demographic information on those patients (e.g., age, gender).

Conduct focus groups and key informant interviews at other sites to determine 1) existing and prior QI and clinical process projects related to early detection of decompensating sepsis (or non-sepsis) patients; 2) barriers to implementing MEWS; and 3) facilitators for implementing MEWS.

Phase 2 (prospective, pre-post): Based on data collected in Phase 1, we will implement a MEWS clinical decision support system at each participating site and determine the impact of this system on key patient outcomes:  direct costs, ICU length of stay, sepsis-related organ injury, hospital length of stay and mortality.

Characteristics of sites who might participate:   

a. Must use an electronic medical record (Utah uses Epic, though we are working on a vendor-neutral technical framework through collaboration with other PaTH sites, including those who use non-Epic EHRs.

b. Must put vital signs (temperature, pulse, respiratory rate, systolic blood pressure) into the electronic medical record

c. Must admit a reasonable volume of patients with sepsis

d. Electronic medical record must be able to fire best practice alerts

e. As temperature, pulse and respiratory rate are not available in the PCORI CDM, must be willing to participate in discussions regarding augmentation of CDM data; our U of U PaTH data architect is helping us develop necessary code to extract this data from EHR.

f. We are investigating potential collaboration within our PaTH network as above

 

Potential funders with RFA/RFP and due dates (if known).

https://grants.nih.gov/grants/guide/pa-files/PA-17-260.html  or

https://grants.nih.gov/grants/guide/pa-files/PA-17-261.html- Plan to submit for January 25, 2017 deadline.  

https://www.pcori.org/funding-opportunities/announcement/pragmatic-clinical-studies-Cycle-3-2017 or

https://www.pcori.org/funding-opportunities/announcement/improving-healthcare-systems-cycle-3-2017 - Plan to submit for February 6, 2018 deadline with LOI due on October 31, 2017

Comments

We use an electronic sepsis trigger tool at Mount Sinai. The tool leads to specific recommended actions, including adequate IVF bolus, immediate abx and lactate levels. These are not prompted via CDSS, which would be a substantial advance if this can be built into the MEWS protocol. 

Hello and thank you for the comment! Our MEWS alert is associated with an order panel that prompts nursing staff to increase VS frequency, obtain blood/urine cultures and serum lactic acid. The MEWS provider BPA prompts an order set with antibiotics and fluid bolus. The provider can opt out, but the sepsis orders are available to make it easier to place them. 

Hi Kencee -- For phase 1 of this study, I agree with Andy's comment from the call today that it would be interesting to evaluate sensitivity/specificity of the MEWS alert at different sites. In phase 2, I think it would be interesting to incorporate some measures of unintended consequences particularly focusing on alert fatigue. I think the preliminary findings you mentioned regarding lack of increase in measured unintended consequences (use of broad-spectrum antbiotics, iincidence of C. diff, and incidence of resistant infections) were quite interesting. It would be helpful to know if those findings also end up being true in a multi-site evaluation.

Our patient engagement team discussed your project earlier today and I have been asked to summarize this discussion. The patient advisors on this team had some specific comments/questions:
  1. What is your patient stakeholder engagement strategy for this project? How can patients, family members and caregivers be involved in all stages of this project - including development, implementation and dissemination? Given the decision–support/electronic health record nature of this project the patient advisors were unsure how exactly how this could be achieved but would welcome your initial thoughts and also would be open to talking with you about this. They noted that they do not want ‘patient engagement’ to be perceived as a barrier but something that can enhance your project. 
  2. Does the University of Utah have Patient & Family Advisory Councils or Patient & Family Centered Care Councils? These could be opportunities to obtain patient stakeholder input. 
  3. Could be an additional component of this study that explored patients' experiences of sepsis within the hospital (has this been described before?). Another thought was that in addition to understanding patients’ experience of sepsis would be to explore caregiver experience.

Hi James, Thank you for these questions and comments. I agree that we need to add more patient centeredness to the proposal. Admittedly, I am new at PCORI research and am meeting with my mentor to work on a more thoughtful reply to your thoughts. I will post those within the next week after we meet.  

Commenting is closed.