Department of Medicine 2016 Tech Challenge

New Uses of Information Technology to Advance the Missions of the Department of Medicine

Real-Time Feedback: Using the Electronic Health Record to Enhance Clinical Reasoning

Idea Status: 

Key team members:

Sirisha Narayana, MD, Alvin Rajkomar, MD, Victoria Valencia, MPH, James Harrison, PhD MPH, Sumant Ranji, MD, Division of Hospital Medicine, UCSF.

Gurpreet Dhaliwal, MD, Department of Medicine, SF Veterans Affairs Hospital.


Regular feedback on patients’ clinical course has long been recognized as a key factor in enhancing physician clinical reasoning and improving collaborative and team-based care (1, 2). The recent Institute of Medicine (IOM) report “Improving Diagnosis in Health Care” cited the lack of clinician feedback on diagnoses as a major contributor to the high prevalence of missed and delayed diagnoses (3). 

Provider surveys demonstrate the challenges in receiving feedback on medical decision-making. In the 2012-2013 ACGME survey, only 40% of UCSF Internal Medicine residents felt they were given data to show their personal clinical effectiveness.  A subsequent needs assessment from the Division of Hospital Medicine revealed that while many residents found value in learning what happened to their patients after discharge very few received this information consistently, either from their own reviews or from their ward attending physicians (4). Key barriers to longitudinal follow-up included time constraints, discontinuous training environments, and difficulty obtaining patient information. Attending physicians agreed that providing housestaff with this information was useful, but were limited by time constraints and the inability to extract the necessary information from the electronic health record in a methodical and efficient manner (4). Attending physician ability to retrieve this information to receive feedback on their own decision-making is likewise limited. 

We are currently employing an educational strategy to improve trainee feedback on clinical decision-making and to encourage self-directed learning: Interns on their patient safety rotation receive electronic lists of all the patients they cared for on inpatient general medicine ward rotations at Moffitt-Long hospital. Lists are generated using Structured Query Language (SQL) queries. They then review the EHR of their patients, guided by a reflection worksheet which provides a structure for the chart review. Lastly, they debrief with their peers at the end of the rotation on lessons they learned through this process. Based on preliminary analysis of this intervention, interns felt that post-discharge patient follow-up was extremely valuable to their professional development. They reported that reviewing patients’ clinical courses would change their future practices by advocating for earlier end-of-life counseling, improving discharge transitions, and adjusting their clinical decision-making while the patient was still hospitalized. 

Ultimately, however, they felt they would not be able to maintain this habit on their own without facilitation from either their training environment or the EHR. The IOM notes the importance of health information technology (IT) innovation in enhancing clinical reasoning in health care. Specifically, the electronic health record (EHR) could be used to automatically generate feedback to clinicians upstream thereby facilitating learning from outcomes of diagnostic decisions (3, 5).

Therefore, we propose developing an electronic method to track patients providers have seen in order to verify decision-making and promote feedback among providers. 

Proposed Intervention:

1. We would first conduct a needs assessment of physician, pharmacy, and nursing providers on their methods of patient follow-up (i.e. EHR review, patient phone call, word-of-mouth, etc).

2. We envision creating an active list within Apex which would auto-populate with the patients on whom a provider has written some form of documentation (i.e. progress note, H&P, discharge summary). This list would consist of the patient name, MRN, age, sex, date of admission, date of discharge, and discharge diagnoses (though these could be adjusted per provider preference). Separate columns would also be listed for number of readmissions (providers could refresh the list to provide the most up-to-date information) and new radiographic or lab results (with a direct link to the Apex inbox). Providers could use this list to conduct chart reviews on patients they have managed and learn from their clinical course. 

3. We would work with our nursing and pharmacy colleagues to determine the best way to depict similar information in their specific Apex view.

4. We would collaborate with IT specialists at the VA and SFGH to discuss the technological nuances that would facilitate a similar project in their EHRs.

This intervention would facilitate self-regulated feedback and enhance opportunities for clinical reasoning assessment.


1. Schiff GD. “Minimizing Diagnostic Error: The Important of Follow-Up and Feedback,” Am J Medicine 2008.

2. Croskerry P. “The Feedback Sanction.” Acad Emerg Medicine, Nov 2000.

3. Balogh EP et al, “Improving Diagnosis in Health Care,” Committee on Diagnostic Error in Health Care; Institute of Medicine, National Academies of Sciences, Engineering and Medicine. September 2015.

4. Gottenborg E. and Ranji, S. “Post-Discharge (and Float) Clinical Feedback & Resident Education,” Division of Hospital Medicine Incubator Presentation, Dec 12, 2013.

5. G. Schiff and D. W. Bates. “Can electronic clinical documentation help prevent diagnostic errors?,” New England J Med. 362(12):1066–1069. 2010.



This is an obvious functionality of the EMR that I have wished existed many times, for everything from needing a case to present in report to wondering if my discharged patient's send-out test ever came back. It would also be a necessary first step to exploring how technology can mitigate the structural factors which limit continuity in training (such as short hospital stays and frequent rotation changes).

Hi Laura,

Thanks for your comment. Yes, the structural aspects of training are a definite issue. We are collaborating with folks at SFGH and the VA to determine the technological capabilities to provide such information. A work in progress!


I love this idea and think there are many ways to help make this happen on the Epic side. We should talk offline about a report in Epic to let providers see all patients they've written notes on in the last x days, and a way to impose a questionnaire on the same.

Thanks, Raman. Looking forward to chatting more about this!

Commenting is closed.