Chief Health AI Officer  |  Executive VP of Physician Services  |  Chief Medical Information Officer
Chief Nursing Informatics Officer  |  Chief Innovation Officer  |  AVC Clinical Research  |  CTSI
Chief Research Informatics Officer  |  BCHSI  |  Division of Clinical Informatics & Digital Transformation
Chief Information Officer  |  Chief Data Officer

Section 1. The UCSF Health Problem 

Problem Statement: Across UCSF Health, nearly 59,000 hospital discharge summary (DCS) narratives are manually written each year. As one of the longest, yet most important forms of clinical documentation, the DCS places substantial documentation burden on inpatient providers. Unfortunately, as both the literature and inpatient provider experience have shown,¹ producing a high quality DCS in a timely manner is challenging. Whether providers are pressed for time, or are simply not aware of all of the details of a patient’s hospital encounter (the discharging physician at UCSF, is on average the last of 3 sequential physicians having cared for a patient, and therefore may not be aware of all events throughout the encounter), physician-written discharge summaries are not error free, as we recently demonstrated in a UCSF study accepted in JAMA Internal Medicine and is in pre-print.² (Figure 1) Poor quality discharge summaries may then have multiple downstream ripple effects impacting subsequent quality of care and patient safety. 

Background: The hospital DCS is an accounting of important hospital events and treatments that must be identified, synthesized, and composed by the discharging provider. A substantial contributor to documentation burden (often described as an epidemic³) is the DCS, a uniquely time intensive source of burden affecting not only all inpatient providers directly, but - depending on the quality of the summary – downstream providers too (e.g. primary care physicians, Skilled Nursing Facility physicians, and subsequent inpatient providers who commonly rely on reviewing the prior DCS when readmitting a patient).

The sequelae of this documentation burden include reduced face-to-face time on inpatient care, increased medical error rates, reduced document quality, physician burnout, and attrition. In one study, over 44% of hospital physicians reported not having sufficient time to compose high quality discharge summaries.¹ Burden that is unique to the DCS derives from the need to review notes, procedures, and events throughout the hospital encounter (the longer the encounter, the more difficult the task); synthesize; reconstruct; and manually compose a problem-by-problem narrative of important hospital events and treatments. The discharging physician is often in the position of trying to reconstruct events that occurred before his or her care for the patient. Furthermore, because the hospital environment is a busy setting, with the physician taking care of multiple patients and being paged on average every 15 minutes (internal study), composing a summary is done either while the physician is on service in a highly interruptive environment - or done after hours in “pajama time.” Ultimately, not only is documentation burden recognized by leading healthcare organizations as a critical systemic problem, but it has also been designated by UCSF Health as a high priority IT Portfolio Initiative for FY2025 here. Additionally, by increasing the efficiency of DCS narrative production, an important secondary benefit of this proposal is helping facilitate discharge readiness earlier in the day, thereby opening inpatient beds earlier and decompressing the number of patients boarding in the emergency department (also a UCSF priority). Therefore, this proposal aligns with several existing UCSF priorities.

Section 2. How might AI help? 

AI assistants are already being piloted in many health systems to reduce documentation burden through ambient scribing (USCF is currently piloting this technology), and AI-generated draft responses to inbox messages, noting statistically significant reductions in burden and burnout.⁴ Although AI’s potential for higher-stakes use cases such as medical decision making is still being explored, LLMs are well-known to excel in lower-stakes use cases such as medical summarization.⁵ We have already demonstrated safety and feasibility of using Versa for generating the DCS narrative by reading through all hospital encounter notes and comparing GPT-4 Turbo’s output to physician-generated summary narratives in a retrospective study that has been accepted for publication by JAMA Internal Medicine. According to processes outlined by the UCSF AI Governance Committee here, the next step following the retrospective analysis we did for the JAMA IM paper, would be a prospective pilot, which we are hereby proposing.

While the proposed pilot targets DCS for the Hospital Medicine service (as in our JAMA IM paper), the value of an LLM-drafted discharge summary has tremendous potential to scale across all inpatient specialty services at UCSF.

Section 3. How would an end-user find and use it?

An end-user would invoke the LLM-generated DCS narrative directly in the existing discharge summary workflow in APeX. (Figure 2A) As discussed with the APEx Enabled Research (AER) team on March 10, 2025, any inpatient provider writing a hospital DCS could optionally launch the LLM simply by following the usual workflow and clicking on “Discharge Summary” that would then be followed by the option to choose an LLM generated draft for provider review. Because Versa is HIPAA-compliant, UCSF is uniquely positioned to do this work and become a national leader in LLMs for the DCS.

Section 4. Embed a picture of what the AI tool might look like (Figure 2)

Figure 2A shows the existing APeX workflow for generating a discharge summary. The inpatient provider selects “Discharge Summary” from within the Discharge Navigator tab, opening a Discharge Summary template. Many of the items in the template (e.g. medication list, referrals, etc.) will auto-populate from APeX. However, the summary narrative (“History (with Chief Complaint)” and “Brief Hospital Course by Problem”) must currently be manually composed by the inpatient provider. This is the section that involves the most documentation burden. Figure 2B shows sample LLM output where the drafted discharge summary narrative would be placed and available for provider review. Furthermore, Epic has the ability to optionally include citations /links associated with LLM-generated statements to information sources from the hospital encounter, thereby facilitating review and verification by the provider.

Section 5. What are the risks of AI errors? 

The proposed use of AI for the DCS involves human-in-the-loop. Namely, inpatient providers must review LLM-drafted DCS much in the same manner as they must review drafted inbox replies to patients. It remains the inpatient provider’s responsibility to ensure the accuracy of the DCS. Nevertheless, in our retrospective study, not only did we find that providers reviewing the summaries (blinded as to whether the summary was LLM- or physician-generated) had equal preference for both (χ2 = 5.2, p=0.27), and that they had similar overall quality ratings (3.67 [SD 0.49] vs 3.77 [SD 0.57]; p=0.674), but that there was no difference in the harmfulness scores associated with each LLM vs. physician error. While accurate medical summarization is important, (we found that physicians also made errors of omission, inaccuracy, and even hallucinations), the stakes of AI use for medical summarization with human-in-the-loop are lower than for AI used in medical decision making. Finally, identifying errors in the LLM-generated content could optionally be accomplished by enabling a reporting button within APeX with which inpatient providers could report potential errors for further investigation and mitigation. We also hope to be one of the first use cases for UCSF’s Impact Monitoring Platform for AI in Clinical Care (IMPACC), which is being built for continuous, automated, longitudinal AI monitoring.

Section 6. How will we measure success? 

There are many potential approaches to deploying the intervention and metrics to measure success. In this high level overview, it is critical that the deployment approach minimizes the friction to existing clinical workflows that could limit adoption and user satisfaction. We therefore propose an observation cohort approach allowing each inpatient provider to have the option to choose an LLM-drafted summary for each patient. Although a randomized controlled trial (RCT) – either randomized at the provider level (certain providers opted in for all of their patients) or at the patient level (same provider could use the LLM drafts for some patients but not others) – would offer the most rigorous evidence, an RCT could cause substantial workflow friction.

Therefore, we will measure success in an observational cohort study on two main domains:

1. Measurements using data already being collected in APeX: Leveraging the APeX audit logs, an area of deep expertise in our research group, we will assess the burden on inpatient providers using the LLM-drafted content by: 1. Measuring the amount of manual editing of the LLM-drafted narrative the provider makes (character count and % of narrative text edited as proxies for burden), and 2. Time savings as measured by the amount of time spent either editing LLM-drafted narratives or composing provider-generated narratives.
2. Measurements not necessarily available in APeX, but ideal to have: In any new technology deployment, user satisfaction is important to capture. If possible, we will attempt to capture user satisfaction with the LLM-generated narrative process by collecting Net Promoter Scores from inpatient providers using the LLM-drafted narratives, as well as those who are downstream recipients (e.g. PCPs, SNF physicians). Also ideal, if possible, would be to capture in APeX, inpatient provider flags of potential errors (inaccuracies, omissions, hallucinations), not only as a measure of quality and safety, but for investigators to identify concerns, and develop mitigation strategies. Optionally, the investigator team can quantify these error types, or (in a more scalable approach) we could potentially embed a multi-agentic platform such as CrewAI (crewai.com) to allow multiple LLMs to check one another for errors.

Section 7. Describe your qualifications and commitment:

This project is led by Dr. Benjamin Rosner, MD, PhD, FAMIA. Dr. Rosner is a hospitalist, a clinical informaticist, an AI researcher within DoC-IT, and the Faculty Lead for AI in Medical Education at the School of Medicine. He has years of experience developing, testing, and deploying digital technologies into healthcare, and he was the lead author of the retrospective study that underpinned the evidence for this solution. Dr. Rosner has worked directly with AER through the Digital Diagnostics and Therapeutics Committee (DD&T) for 6 years.

Charu Raghu Subramanian, MD is a Clinical Informatics Fellow, a hospitalist, and a co-lead author on the retrospective research study that underpinned the evidence for this pilot. She holds certifications in Epic Clarity and as an Epic Builder.

Citations 

1.    Sorita A, Robelia PM, Kattel SB, et al. The ideal hospital discharge summary: A survey of U.S. physicians. J Patient Saf. September 6, 2017. doi:10.1097/PTS.0000000000000421

2.    Williams CYK, Subramanian CR, Ali SS, et al. Physician- and Large Language Model-Generated Hospital Discharge Summaries: A Blinded, Comparative Quality and Safety Study. medRxiv. September 30, 2024. doi:10.1101/2024.09.29.24314562 https://www.medrxiv.org/content/10.1101/2024.09.29.24314562v1 (also accepted for publication in JAMA-IM, with anticipated publication in May, 2025)

3.    Hobensack M, Levy DR, Cato K, et al. 25 × 5 Symposium to Reduce Documentation Burden: Report-out and Call for Action. Appl Clin Inform. 2022;13(2):439-446. doi:10.1055/s-0042-1746169

4.    Garcia P, Ma SP, Shah S, et al. Artificial Intelligence-Generated Draft Replies to Patient Inbox Messages. JAMA Netw Open. 2024;7(3):e243201. doi:10.1001/jamanetworkopen.2024.3201

5.    Van Veen D, Van Uden C, Blankemeier L, et al. Adapted large language models can outperform medical experts in clinical text summarization. Nat Med. 2024;30(4):1134-1142. doi:10.1038/s41591-024-02855-5

Summary of Open Improvement Edits

• Added suggested measurement concepts to allow inpatient providers to flag potential errors (inaccuracies, omissions, hallucinations)
• Added option for investigator team to quantify errors
• Added option for multi-agentic platform to identify and quantify errors
• Added Medrxiv link to manuscript in citation #2 (to be published in JAMA-IM in May, 2025)

The UCSF Health Problem: Hospital Acquired Pressure Injury (HAPI) is a preventable injury to skin or soft tissue that is acquired during a patient’s hospital stay. Reducing HAPI rates is a top priority for UCSF Health leadership as the occurrence of HAPI is detrimental to patient experience and outcomes, results in significant costs (estimated cost to the health system for 1 HAPI is $18,000-$27,000), and is a critical quality measure in the evaluation of hospital performance. The creation of interdisciplinary workflows, utilization of structured problem-solving, and electronic dashboards to monitor HAPI rates and risk have reduced HAPI rates at Benioff Children’s Hospital (BCH) by 64% over the last 15 months. However, poor HAPI bundle compliance continues to be a driver of HAPI as current HAPI rates are 27% higher than national benchmarks. Critical care units account for 95% of HAPI at BCH.

The Gap: There is a lack of real-time electronic medical record (EMR) tools that assist interdisciplinary bedside teams to more effectively support efforts to prevent HAPI.

The objective of this proposal is to develop an AI report to reduce HAPI in critical care units at BCH (Oakland and SF) by improving bundle compliance through an interdisciplinary approach.

Generalizability: The AI report can be leveraged to support HAPI prevention and management across UCSF Hospitals (including adult patients) and could also be adapted to prevent other top harms (e.g Catheter Associated Blood Stream Infections, etc).  

How might AI help? Bedside clinical teams document hundreds of clinical observations related to evidence-based risk factors for HAPI prevention daily in flowsheet rows and notes for each patient, making it challenging to assess if care provided meets expected standards. For example, routine aspects of patient care (such as nutrition delivery) are often disrupted due to procedures, feeding intolerance, or other factors, leading to gaps in knowledge of actual vs ideal state of delivered nutritional goals. Lastly, there are knowledge gaps regarding how to modify care goals when patients with risk factors for developing HAPI are identified.

Multimodal generative AI is ideal for creating a report that can summarize clinical data because it can seamlessly process and integrate diverse data types—such as structured data and free-text notes—into a unified model. By combining techniques like natural language processing for unstructured text and machine learning for structured data, the AI report can extract meaningful insights across all data formats. Importantly, its generative capabilities enable it to summarize clinical data and propose actionable content (such as tailored recommendations) in real-time.

Data Sources: All data elements requested are either documented in flowsheet rows or templated notes in Apex at the individual patient level. Data requested for this report are based on published, validated risk factors for HAPI development. Structured Data: Nutrition (e.g. formula or TPN prescribed, rate of formula/TPN delivery, hours over which nutrition delivered, regular diet order, or percent of regular diet finished, HAPI bundle elements (e.g. repositioning (time of turn and position in which patient is repositioned), perfusion, skin hygiene, mobility promotion, device rotation, application of barrier creams, etc), Medication (e.g. medication administration of vasoactive medication, medications administered through central lines), Medical Devices/LDA (e.g. endotracheal tube, noninvasive positive pressure device, etc).  Unstructured Data: Nutrition: Goal nutritional intake (found in templated registered dietician note), HAPI injury status: Found in templated wound care notes.

How would an end-user find and use it?

Location: The AI report will be found in the “Summary Tab” as nursing, respiratory therapy, and physicians/advanced practice providers (APP) routinely access this tab when viewing patient charts in the inpatient setting. The proposed alert to remind bedside to staff to perform key HAPI bundle components will trigger when a bundle element is overdue when a patient’s chart is opened.

Timing of Support: Support will be most effective in real-time, as HAPI prevention efforts are carried out as frequently as every two hours in critical care units. Reminders to comply with unit policies (bundle assessment/documentation) and suggested recommendations will be made available as the need for improvement is identified.

What end users may see? The AI tool will generate a report summarizing compliance with best practices for the current day and the past week. It will also provide recommendations to improve bundle compliance. This alert can be temporarily silenced if clinical stability prevents compliance. For physicians and APPs, a suggestion box with action items will appear when they click on the hyperlink under the suggested interventions section. For immediate action items (e.g., placing a wound care consult), an alert will appear as soon as the chart is opened.

How are recommendations explained: The non-compliant bundle element and the reason for non-compliance will be visually highlighted, along with a brief suggestion for getting back into compliance. Some interventions need only brief explanations (e.g unit policy is to turn patient every 2 hours, but it has been 4 hours since last turn). For more nuanced suggestions (such as optimizing nutrition), the report will describe why and length of time nutrition is not at goal and propose actionable suggestions (such as possible means for nutritional delivery).

Picture of Embedded AI Tool:

What are the risks of AI error? 

AI errors could be caused by missing data and hallucinations. Missing data could lead to both under and overestimation of HAPI risk. However, because a key component of this report is to increase bundle compliance (and by association documentation of bundle compliance), we plan to mitigate errors from missing data by improving data entry. To validate the AI model, domain experts will regularly review its outputs and compare them with existing data reports (HAPI dashboard, nurse audit data) and analysis of component data (e.g., skin assessment, patient positing, etc) through clarity queries. Additionally, accuracy of information extracted from note text will be assessed by comparing AI output of notes with text matching (text mined from templated notes [existing standard]). The project lead has extensive experience analyzing structured and unstructured EMR data (Mahendra et al, Pulm Circ, 2023; Mahendra et al, Crit Care Explor, 2021). A feedback button will also be available for end-users to report inaccuracies. 

Feedback: A feedback button will be available for end-users to report inaccuracies at any time in the EMR.  Additionally, we will build an active user-centered design process working group to seek and incorporate iterative input from end users as the report is developed and implemented in the clinical setting. Feedback will be used to 1. Continuously retrain and refine the model for improved accuracy and reliability 2. Optimize bedside use of this tool 3. Identify other factors (staffing, equipment availability, etc) that may be identified as important factors to monitor to improve HAPI prevention effectiveness. 

How will success be measured? Success will be defined by both outcome and process measures. The primary outcome measure will be BCH Stage 2 and greater HAPI rates/1,000 patient days. The goal would be to realize a sustained reduction of HAPI rates to below national benchmark data across BCH. HAPI rates are readily available as they are closely monitored across UCSF Health in the zero-harm dashboard. The secondary outcome is improved adherence to the HAPI prevention bundle (a key measure of adoption and utilization of the tool). Documentation of bundle compliance will be assessed through query of structured flowsheet row data and in person audits (routinely performed by nursing and documented in the digital rounding tool). An automated dashboard is also being built to report bundle compliance by UCSF Health IT.  

Qualifications: Malini Mahendra MD (project lead) is a pediatric intensivist and data scientist. As a certified clarity data analyst with expertise in use of machine learning and natural language processing algorithms, she has extensive experience analyzing UCSF EMR data for quality improvement.  She is also the local quality improvement and informatics lead for the mission bay pediatric intensive care unit.

Deborah Franzon MD, MHA (collaborator) is the Executive Medical Director for Quality and Safety at BCH with over 20 years of clinical expertise and implementation science experience.   Dr. Franzon has been the recipient of prior UCSF open proposal awards including: Learning Health System Innovations (2017-2019) building a machine learning predictive model for extubation in PICU patients, soon to be integrated into Epic; and Caring Wisely (2021) to reduce delirium in PICU patients across BCH SF and Oakland.  She developed, implemented and published on the impact of an EHR enhanced dashboard in reducing CLABSI rates in the PICU. (Pediatrics, 2014) She has a proven track reducing harm, improving clinical outcomes, and driving data-informed, innovative change through collaborative, patient-centered leadership.

This project is supported by BCH Leadership: Nicholas Holmes MD MBA (President of BCH), Joan Zoltanski MD MBA (Chief Medical Officer of BCH), Judie Boehmer RN MN (Chief Nursing Officer BCH), Michael Lang MD (Chief Medical Information Officer for Children's Services), Jeff Fineman MD (Pediatric Critical Care Division Chief), Shan Ward MD and Loren Sacks MD (MB Pediatric ICU and Pediatric Cardiac ICU Medical Directors), Mary Nottingham RN and Lori Fineman RN (MB Clinical Nurse Specialists, Pediatric Critical Care and Pediatric Cardiac Critical Care), Mandeep Chadha MD (BCH-Oakland Pediatric Critical Care Quality Lead).

Selected References:

Mahendra M, Chu P, Amin EK, Nawaytou H, Duncan JR, Fineman JR, Smith-Bindman R. Associated radiation exposure from medical imaging and excess lifetime risk of developing cancer in pediatric patients with pulmonary hypertension. Pulm Circ. 2023 Aug 21;13(3):e12282. doi: 10.1002/pul2.12282. PMID: 37614831; PMCID: PMC10442605.

Mahendra M, Luo Y, Mills H, Schenk G, Butte AJ, Dudley RA. Impact of Different Approaches to Preparing Notes for Analysis With Natural Language Processing on the Performance of Prediction Models in Intensive Care. Crit Care Explor. 2021 Jun 11;3(6):e0450. doi: 10.1097/CCE.0000000000000450. PMID: 34136824; PMCID: PMC8202578.

Pageler NM, Longhurst CA, Wood M, Cornfield D, Suermondt J, Sharek P, Franzon D. Use of electronic medical record-enhanced checklist and electronic dashboard to decrease CLABSIs. Pediatrics. 2014;133(3):e738-e746. doi:10.1542/peds.2013-2249

Solutions for Patient Safety Operational Definition and Prevention Bundle. (rev. 2020). https://static1.squarespace.com/static/62e034b9d0f5c64ade74e385/t/636177d9b97cd87674b58f18/1667332057726/PI-Bundle-Op+Def.pdf

Padula WV, Delarmente BA. The national cost of hospital-acquired pressure injuries in the United States. Int Wound J. 2019;16(3):634-640. doi:10.1111/iwj.13071

Johnson AK, Kruger JF, Ferrari S, et al. Key Drivers in Reducing Hospital-acquired Pressure Injury at a Quaternary Children's Hospital. Pediatr Qual Saf. 2020;5(2):e289. Published 2020 Apr 7. doi:10.1097/pq9.0000000000000289

The UCSF Health problem

Falls among hospitalized adult patients are a significant patient safety concern, associated with higher morbidity, prolonged hospital stays, and approximately $62,000 in non-reimbursable costs per fall.¹ While our fall rate is below the national benchmark, falls are considered a never event.²Moreover, they are a quality indicator under UCSF Quality and Safety True North Pillar. Nurses play a critical role in fall prevention³ and are the intended end-users of our proposed AI-Augmented Fall Prevention Tool.

UCSF Health currently employs an evidence-based Fall Prevention Program that incorporates Fall TIPS⁴, including risk assessment, care planning, and the delivery of tailored interventions. Nurses assess and document fall risk every shift using the STRATIFY tool⁵, which is based on 5 features: recent history of falls, and observations agitation, visual impairment, toileting, and mobility. However, STRATIFY uses a fixed threshold for identifying “high risk” patients, which limits its utility in rapidly changing clinical situations. Additionally, many of its components are already documented in other nursing assessments but are not easily integrated with other data.

A UCSF pilot in 2020 evaluated the Epic AI Fall Risk Prediction model⁶ with 38 variables, but it did not outperform STRATIFY and was not adopted into the workflow. It also lacked actionable recommendations for end-users, limiting its utility in patient care. Despite this, AI remains a promising avenue for integrating broader clinical features into fall risk prediction.

Major limitations to the current fall prevention approach are 1) the over-reliance on an overall risk score rather than attention to individual factors, 2) the high cognitive load to assess the contribution of the medical history and recent changes in status to inform tailored care planning, 3) and the burden of documentation that takes away from direct patient care.

How might AI help?

The primary goal of the proposed AI-Augmented Fall Prevention Tool is to reduce inpatient falls through timely and targeted prevention. By leveraging real-time clinical data, the tool will assist nurses with care planning by providing tailored, actionable recommendations. It will also reduce documentation burden by synthesizing existing information into a user-friendly format. The tool will incorporate both risk prediction and large language models and will be capable of reading from and writing to APeX. The tool will consist of a risk prediction and clinical decision support system with an integrated end-user feedback loop (Figure 1) These interdependent components will be built across three phases.

Phase 1: The Risk Prediction model will generate a dynamic profile that presents 1) a risk score and corresponding stratification (e.g., high, medium, low) and 2) the clinical features underlying fall risk. We will use APeX and Incident Report (IR) data to label cases. Fall events are documented in a separate IR database, but as of January 2024, they are also documented in APeX in the Post-Fall Flowsheet and Note. Since January 2022, there have been 1,148 documented falls.

Predictive features will be selected based on research evidence^7,8,9 and their availability in APeX. Our prior work has shown that nurses already document STRATIFY elements in other daily flowsheets. In addition to pulling these data, we will include clinical, environmental, and time-dependent variables such as time of day, unit type, demographics, medical history, admission assessments, vital signs, medications, lab values, new diagnoses/procedures, and functional status. The model will automatically update the risk profile as new data become available.

We will begin by focusing on structured data from the Admission Navigator, Diagnoses, Problem List, Medication List/MAR, and Flowsheets. We will later explore the added value of unstructured data (e.g., gait assessments in Physical Therapy (PT) notes) and the technical costs associated with their inclusion.

 How would an end-user find and use it?

Phase 2: The Clinical Decision Support System (CDSS) will be embedded in the patient’s chart.Nurses would interact with it at the beginning of the shift and when a patient’s clinical status changes. The tool will visualize each patient’s fall risk, display trendlines, and highlight top contributors to any change. A clinical advisory will appear when a new high-risk patient is identified or when a patient’s risk sharply increases. It would recommend tailored interventions or adjustments to the current Falls Care Plan based on research evidence^10,11. Recommendations will incorporate patient preferences and conditions (e.g., language, visual impairment) and be presented in natural, LLM-generated language to ensure clarity and increase nurses trust in the AI.

Phase 3 will include End-user Feedback and a Tracking Dashboard.  The nurse will be able to accept or reject the clinical advisory recommendation. If accepted, the AI tool would automatically update the Care Plan, reducing the need for manual documentation. It would also generate a reminder to document the performed intervention by the end of shift, if not done already. If the nurse rejects the advisory based on their clinical judgement, they can provide a rationale using a standardized list of options that reduce the cognitive burden. This input will support reinforcement learning with a "nurse in the loop" model. A Tracking Dashboard built in Tableau will display model and user metrics, monitored by the Falls Committee. This interprofessional committee, which reviews all fall incidents, has been actively involved in developing this proposal.

Key stakeholders — including Fall TIPS leaders, Falls Champions, and nurse informaticists — will co-design the CDSS interface to ensure usability, minimal clicks, and integration into workflows. We will apply the Consolidated Framework for Implementation Research¹² and the NASSS model (Non-adoption, Abandonment, Scale-up, Spread, and Sustainability)¹³ to guide adoption and sustainability.

What are the risks of AI errors?

Primary risks include false positives and false negatives. False positives may flag too many patients as high risk and lead to alert fatigue from unnecessary clinical advisory alerts. False negatives may potentially increase falls due to missed opportunities for fall prevention. We plan to track model fit statistics to reduce these risks. Additionally, we will review clinical advisory rejections, which may suggest excessive false positives, and their rationale. We aim to mitigate these issues by optimizing the thresholds, refining the model, and engaging our stakeholders. Moreover, to reduce bias, we will plan to periodically recalibrate the model across age, race/ethnicity, and diagnosis groups.

How will we measure success?

This multi-phase project requires significant clinical and technical investment, but it has the potential to improve patient safety, documentation efficiency, and reduce EHR-related burnout — offsetting its costs. It may also be extended to other preventable harms and serve as a model for nursing-led AI innovation. Lastly, it would generate pilot data for a larger extramural grant.

Aim 1: Evaluate the risk prediction model fit and compare its ability to identify high risk patients to STRATIFY. Model metrics will include False Positive Rate, False Negative Rate, Positive Predictive Value, and falls occurring in patients classified as “low risk”. We will adopt the model if it performs as well or better than STRATIFY. If successful, we propose to replace STRATIFY with this AI tool and embed it in the current Fall TIPS program.

Aim 2: Evaluate and continuously refine the user experience with brief surveys and focus groups with nurses as well as general AI tool usage with APeX metadata. We will also use APeX metadata to measure the change in time spent on fall related documentation.

Aim 3: Evaluate the impact on the number of inpatient falls per 1,000 patient days, and the rate of falls with injury. These measures are available through the Zero Harms Dashboard. We will consider abandoning the project if there is a persistent increase in falls especially in patients that AI flagged as “low risk”, high advisory rejection, or low user satisfaction. The thresholds for these outcomes will be decided upon with the stakeholders.

• A list of measurements using data that is already being collected in APeX: AI toolusage (views of the risk profile, clickthrough rate on advisory recommendations, advisories accepted and rejected, rationale for rejection), documentation (time spent on fall related documentation, time to the completed documentation after the advisory reminder).
• A list of other measurements you might ideally have to evaluate success of the AI: nurse satisfaction and trust in the AI tool (focus groups), nurse agreement with AI-generated explanations (surveys/focus groups), use of fall prevention resources (e.g., safety attendants, PT consults), rates of falls and falls with injury, and potential cost savings.

Qualifications and commitment

The project leads are Maria Yefimova, PhD, RN (5% effort) and Sasha Binford, PhD, MS, RN, PHN, AGCNS-BC (5% effort). Both are faculty in the Department of Physiological Nursing at UCSF School of Nursing. Dr. Yefimova is the lead nurse scientist with UCSF Health. With her implementation science background and experience evaluating remote patient monitoring programs, she will oversee the development and evaluation. Dr. Binford is the Nursing Clinical Quality Specialist. She is the Fall Subject Matter Expert and has served as the tri-Chair of the Falls Committee. Previously, she has led the development of the UCSF AI delirium risk prediction model. She will contribute her expertise on the model development and validation and will liaison with clinical and operational stakeholders.

Falls Committee members that include Clinical Nurse Specialists (Melissa Lee MS, RN, PCCN, GCNS-BC, NEA-BC, chair), Nursing Quality (Meghan Sweis, MSN, RN, CNL, CPHQ), and Continuing Quality Improvement (Adam Cooper, DNP, RN, NPD-BC, EBP-C), as well as Nursing Informatics (Kay Burke, MBA, BSN, RN, NE-BC) will provide in-kind effort on model validation and evaluation.

References

1. Dykes, P. C., Carroll, D. L., McColgan, K., Hurley, A. C., Lipsitz, S. R., & Bates, D. W. (2023). Cost of inpatient falls and cost-benefit analysis of implementation of an evidence-based fall prevention program. JAMA Health Forum, 4(1), e225125. https://doi.org/10.1001/jamahealthforum.2022.5125
2. Centers for Medicare & Medicaid Services. (2006, May 18). Eliminating serious, preventable, and costly medical errors - never events.
3. Horta, R. S. T. (2024). Falls prevention in older people and the role of nursing. British Journal of Community Nursing, 29(7), 335–339. https://doi.org/10.12968/bjcn.2024.0005
4. Dykes, P. C., Carroll, D. L., Hurley, A. C., Lipsitz, S., Benoit, A., Chang, F., Meltzer, S., Tsurikova, R., Zuyov, L., & Middleton, B. (2010). Fall prevention in acute care hospitals: A randomized trial. JAMA, 304(17), 1912–1918. https://doi.org/10.1001/jama.2010.1567
5. Oliver, D., Britton, M., Seed, P., Martin, F. C., & Hopper, A. H. (1997). Development and evaluation of evidence-based risk assessment tool (STRATIFY) to predict which elderly inpatients will fall: case-control and cohort studies. BMJ, 315(7115), 1049–1053. https://doi.org/10.1136/bmj.315.7115.1049
6. Cognitive Computing Model Brief: Inpatient Risk of Falls Epic Systems Corporation.
7. Appeadu, M. K., & Bordoni, B. (2023). Falls and fall prevention in older adults. In StatPearls. StatPearls Publishing. Retrieved April 1, 2025, from https://www.ncbi.nlm.nih.gov/books/NBK560761/
8. Mao, A., Su, J., Ren, M., Chen, S., & Zhang, H. (2025). Risk prediction models for falls in hospitalized older patients: A systematic review and meta-analysis. BMC Geriatrics, 25, Article 29. https://doi.org/10.1186/s12877-025-05688-0
9. Ghosh, M., O’Connell, B., Afrifa-Yamoah, E., Kitchen, S., & Coventry, L. (2022). A retrospective cohort study of factors associated with severity of falls in hospital patients. Scientific Reports, 12, 12266. https://doi.org/10.1038/s41598-022-16403-z​
10. Pierre-Lallemand, W., Coughlin, V., Brown-Tammaro, G., & Williams, W. (2025). Nursing-led targeted strategies for preventing falls in older adults. Geriatric Nursing. https://doi.org/10.1016/j.gerinurse.2025.02.005
11. Ojo, E. O., & Thiamwong, L. (2022). Effects of nurse-led fall prevention programs for older adults: A systematic review. Pacific Rim International Journal of Nursing Research, 26(3), 415–429. https://he02.tci-thaijo.org/index.php/PRIJNR/article/view/258061
12. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. https://doi.org/10.1186/1748-5908-4-50
13. Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A’Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017). Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. Journal of Medical Internet Research, 19(11), e367. https://doi.org/10.2196/jmir.8775

Section 1. The UCSF Health problem. 

• Vaginal bleeding with a positive pregnancy test is a common reason for emergency department visits.¹ The most critical concern in this context is ectopic pregnancy (EP), which occurs in 1–2% of pregnancies and, if undiagnosed, can result in serious morbidity and loss of fertility.^2,3 Diagnosis is challenging because the presenting symptoms are non-specific and often overlap with other early pregnancy complications.⁴ Evidence-based clinical guidelines recommend early use of transvaginal ultrasound (TVS) and serial β-hCG testing to identify intrauterine pregnancy (IUP) or EP.¹ However, when the pregnancy location cannot be determined on initial presentation—a situation termed Pregnancy of Unknown Location (PUL)—further testing and follow-up are required to reach a final diagnosis.⁵

Numerous diagnostic algorithms have been proposed to improve classification and outcomes for women with PUL.⁶ While a 2006 consensus statement laid the foundation for PUL management, inconsistencies remain due to varying definitions and classifications of at-risk populations and outcome categories.¹ More recently the M6 prediction model was developed and modified to include BhCG, ultrasound, and clinical characteristics with and without progesterone measures. This model has been published and externally validated, serving as a starting point for this proposal. (REF) A renewed effort is needed to standardize and streamline approaches and improve real-world implementation (including diagnosis and management) of evidence-based PUL care.⁴

At UCSF/ZSFG, a robust clinical guideline for the management of Pregnancy of Unknown Location (PUL) was updated in 2024 and is currently used by residents and attendings to guide risk assessment, triage, and follow-up decisions (ZSFG PUL Guidelines, 2024). Despite this evidence base, implementation remains manual, variable, and burdensome, as exploratory conversations with residents and attendings uncovered, right now:

• Gyn interns rotate every 5 weeks and spend 1–3 hours daily managing a “floater list” of 15–30+ PUL patients with supervision by a daily rotating gyn attending physician.
• Follow-up plans are documented in free text in APeX and vary based on attending style, leading to inefficiencies and inconsistencies.
• There is no embedded clinical decision support, no structured tool for follow-up tracking, and no patient-facing communication platform. Interns spend hours calling floater list patients daily
• Management depends on interpretation of lab results ultrasound and evolving history, making room for error and delayed diagnosis and intervention for this high-stakes condition.

Section 2. How might AI help? 

Recent developments in artificial intelligence (AI) and machine learning (ML) have shown strong potential to improve PUL management, particularly in outcome prediction, risk stratification, ultrasound interpretation, and individualized care. Key advances include:

• Risk-prediction models such as M6 and modified M6 with clinical characteristics, which use serial β-hCG with and without progesterone values to estimate likelihood of ectopic pregnancy, have demonstrated high diagnostic accuracy (AUCs up to 0.89).^7–10 More complex models—such as neural networks and support vector machines trained on clinical, lab, and imaging data—have shown comparable or better performance.¹¹
• These models now power decision-support algorithms, including the validated two-step protocol (progesterone + M6), which reduces unnecessary follow-up while maintaining high sensitivity for ectopic detection.^6,12
• AI in ultrasound image interpretation has been piloted using deep learning to identify subtle features of ectopic pregnancies that may be missed by less experienced clinicians.^13,14
• AI is also enabling personalized management strategies, including expectant management for stable ectopic cases.^6,10,12

Despite this promising evidence base, AI tools for PUL remain largely absent from routine clinical workflows and are rarely integrated with patient-facing communication systems. UCSF has the opportunity to lead in this space for this complex high-stakes condition often requiring weeks of individual follow-up, by developing an embedded AI tool built on its rich historical EHR data, leveraging the existing M6 model, and up to date care algorithm— enhancing efficiency, accuracy, and equity in PUL care.

We propose TRACE (Triage, Risk Assessment, and Communication Engine) a 3-in-1 AI tool embedded in APeX and MyChart to support both clinicians and patients across the PUL pathway:

1. Prediction Model: We will train a machine learning model—building on the published and validated M6 prediction model— working closely with UCSF data scientists for model validation. We will use data from the UCSF EMR following the procedures described in the publications referenced.  We will identify cases using the ICD 10 diagnosis code for PUL and extract data on model inputs including BhCG, ultrasound results, and clinical characteristics (risk factors, symptoms, sociodemographics) for our population to assess the ability of the model to predict ectopic pregnancy, viable intrauterine pregnancy, miscarriage, or persistent PUL. Initially, we expect to replicate the model using only the initial visit and the first follow-up, however, going forward we hope to improve the predictive model utilizing information from all the follow-up visits so that it can inform decisions (such as when to return for the next visit/lab test/US) at each visit as described in step 2 below.
2. Decision Support + Triage Algorithm: We will translate the ZSFG PUL Guidelines (2024 update) into a structured AI-driven triage algorithm, which will be updated with the prediction model results. The tool will generate next-step clinical recommendations based on predicted outcome and patient-specific factors. Example outputs: “Repeat hCG in 48 hours”, “Schedule transvaginal ultrasound 7 days”, “Consider methotrexate”, “Continue expectant management with safety counseling”
3. Patient-Facing Communication Assistant: A chatbot within MyChart will deliver timely, consistent, and language-accessible communication to patients, reducing follow-up delays and clinical workload:

• Confirm follow-up appointments and lab tests
• Notify patients of test results and recommended next steps
• Deliver safety information and education (e.g., signs of ectopic pregnancy)
• Provide reminders and check-ins

This end-to-end system will augment the current time intensive manual “floater list” process, creating a centralized, streamlined, consistent, and patient-centered workflow for PUL management at UCSF.

Section 3. How would an end-user find and use it?

The TRACE AI tool would be most useful when the gyn team is called to consult on a patient with a positive pregnancy test and early pregnancy symptoms (e.g., bleeding, pain), either in the emergency department or outpatient settings (a frequent occurrence). When the consulting gyn clinician initiates hCG testing or flags a case as suspected Pregnancy of Unknown Location (PUL), the AI tool would automatically populate a PUL Management Panel in the APeX EHR.

As shown in Figure 1 (left panel), the tool would provide:

• A predicted outcome classification (ectopic, intrauterine pregnancy, miscarriage, or persistent PUL)
• A risk level (e.g., 43% ectopic), generated by the model
• A visual timeline of serial hCG and US results
• Management recommendations based on embedded clinical guidelines (e.g., "Repeat hCG in 48 hrs," "Order transvaginal ultrasound")
• Action buttons to pend orders or insert a templated clinical note

The patient-facing chatbot interface (see Figure 2, right panel) would operate in parallel via MyChart. Patients would receive:

• Timely updates about lab results and next steps
• Appointment scheduling assistance
• Safety warnings (e.g., symptoms of ectopic rupture)
• Follow-up reminders and options for clarification

Both interfaces are designed to reduce cognitive burden on providers, support standardized care, and ensure that patients-particularly those at high risk-receive clear, timely, and actionable information.

Section 4. Embed a picture of what the AI tool might look like. 

Section 5. What are the risks of AI errors? 

• False negatives, such as classifying a high-risk ectopic pregnancy as low-risk, could result in delayed or missed diagnosis and potentially serious patient harm.
• False positives, such as overclassifying a low-risk case, could lead to unnecessary labs, visits, or patient anxiety.
• Equity risks are possible if the model underperforms for subpopulations (e.g., patients with limited English proficiency, Medicaid insurance, or historically marginalized groups) due to biased training data.

These risks will be mitigated through the following approaches:

• Begin with validated models (e.g., M6) and retrain using UCSF historical data
• Evaluate and report model performance stratified by race/ethnicity, language, and insurance status
• Keep AI recommendations advisory, allowing clinician discretion
• Include transparent outputs and user-facing explanation of AI logic (e.g., “based on hormone trends and symptoms, this is considered high risk”)
• Build an audit dashboard to track tool usage, override frequency, and outcome correlation

Section 6. How will we measure success? 

We will draw on our expertise as implementation researchers to evaluate the effectiveness of this AI tool and will seek additional funding to do so. Prior to introduction we will survey residents and conduct retrospective chart review examine measures described below before and after introduction. We will estimate time demands for both clinicians and patients prior to and after introduction. We will assess whether end-users (clinicians and patients) are using the system as intended, whether it leads to improved decision-making and workflow efficiency, and whether it improves timely and accurate diagnosis of PUL. We will also examine whether the tool supports equity, reduces unnecessary follow-ups, and minimizes risk of delayed ectopic diagnosis.

We will monitor adoption, usage patterns, and clinical outcomes using a combination of APeX-derived metrics and ideal supplemental measures at ZSFG. These metrics will help determine whether to continue expanding the tool within UCSF or to modify or abandon implementation.

A. Measurements using data that is already being collected in APeX:

• Time from first positive pregnancy test to definitive PUL diagnosis
• Number of hCG draws, pelvic ultrasounds, and follow-up visits per PUL patient
• Proportion of ectopic pregnancies diagnosed before rupture
• Proportion of PUL patients receiving timely follow-up
• Frequency of AI tool usage by eligible clinicians (e.g., OB/Gyn residents, Gyn attendings, ED providers)
• Frequency with which AI-generated recommendations are followed vs. overridden
• Number and type of orders pended via the AI interface
• MyChart chatbot message delivery and open rates

B. Additional measurements ideally needed to evaluate success of the AI:

• Clinician-reported satisfaction, time savings, trust, and perceived utility (via surveys or focus groups)
• Patient understanding of follow-up instructions (e.g., short MyChart-based surveys)
• Time spent by interns managing the floater list before and after implementation
• Disparities in model performance and outcomes stratified by insurance status, language, and race/ethnicity
• Rate of errors or safety concerns identified through audit (e.g., missed follow-up for high-risk cases)
• Longitudinal reduction in unnecessary follow-up testing for low-risk cases

Success would be indicated by high tool adoption, improved timeliness and consistency of PUL management, reduced follow-up burden for clinicians and patients, cost-effectiveness, and equitable model performance across subgroups. Low adoption, significant workflow disruption, or evidence of biased or unsafe predictions would be grounds for modification or discontinuation. 

Qualifications and commitment.

This project will be led by Dilys Walker, MD, FACOG.  Dr Walker is a practicing obstetrician gynecologist at ZSFG and a member of the leadership team at the Bixby Center for Global Reproductive Health. Her expertise is in implementation research across the life course. Dr. Walker has decades of experience managing PUL in various settings. She was awarded a Gates Grand Challenges and J and J grant to build and test a Virtual Mentor chatbot for postpartum hemorrhage. Her team has attended 3 works-in-progress sessions with the AER team.

References

1.         Barnhart KT. Ectopic Pregnancy. N Engl J Med. 2009;361(4):379-387. doi:10.1056/NEJMcp0810384

2.         ACOG Practice Bulletin No. 193: Tubal Ectopic Pregnancy - PubMed. Accessed April 1, 2025. https://pubmed.ncbi.nlm.nih.gov/29470343/

3.         Marion LL, Meeks GR. Ectopic pregnancy: History, incidence, epidemiology, and risk factors. Clin Obstet Gynecol. 2012;55(2):376-386. doi:10.1097/GRF.0b013e3182516d7b

4.         Kirk E, Papageorghiou AT, Condous G, Tan L, Bora S, Bourne T. The diagnostic effectiveness of an initial transvaginal scan in detecting ectopic pregnancy. Hum Reprod Oxf Engl. 2007;22(11):2824-2828. doi:10.1093/humrep/dem283

5.         Condous G, Kirk E, Lu C, et al. Diagnostic accuracy of varying discriminatory zones for the prediction of ectopic pregnancy in women with a pregnancy of unknown location. Ultrasound Obstet Gynecol Off J Int Soc Ultrasound Obstet Gynecol. 2005;26(7):770-775. doi:10.1002/uog.2636

6.         Bobdiwala S, Christodoulou E, Farren J, et al. Triaging women with pregnancy of unknown location using two-step protocol including M6 model: clinical implementation study. Ultrasound Obstet Gynecol Off J Int Soc Ultrasound Obstet Gynecol. 2020;55(1):105-114. doi:10.1002/uog.20420

7.         Maheut C, Panjo H, Capmas P. Diagnostic accuracy validation study of the M6 model without initial serum progesterone (M6NP) in triage of pregnancy of unknown location. Eur J Obstet Gynecol Reprod Biol. 2024;296:360-365. doi:10.1016/j.ejogrb.2024.03.010

8.         Kyriacou C, Ledger A, Bobdiwala S, et al. Updating M6 pregnancy of unknown location risk-prediction model including evaluation of clinical factors. Ultrasound Obstet Gynecol Off J Int Soc Ultrasound Obstet Gynecol. 2024;63(3):408-418. doi:10.1002/uog.27515

9.         Christodoulou E, Bobdiwala S, Kyriacou C, et al. External validation of models to predict the outcome of pregnancies of unknown location: a multicentre cohort study. Bjog. 2021;128(3):552-562. doi:10.1111/1471-0528.16497

10.       Hou L, Liang X, Zeng L, Wang Q, Chen Z. Conventional and modern markers of pregnancy of unknown location: Update and narrative review. Int J Gynaecol Obstet Off Organ Int Fed Gynaecol Obstet. 2024;167(3):957-967. doi:10.1002/ijgo.15807

11.       Rueangket P, Rittiluechai K, Prayote A. Predictive analytical model for ectopic pregnancy diagnosis: Statistics vs. machine learning. Front Med. 2022;9:976829. doi:10.3389/fmed.2022.976829

12.       Jurman L, Brisker K, Ruach Hasdai R, et al. Enhancing decision-making in tubal ectopic pregnancy using a machine learning approach to expectant management: a clinical article. BMC Pregnancy Childbirth. 2024;24:825. doi:10.1186/s12884-024-07035-4

13.       Training and testing performance of ectopic pregnancy prediction model... ResearchGate. Accessed April 1, 2025. https://www.researchgate.net/figure/Training-and-testing-performance-of-...

14.       An automated ectopic pregnancy prediction system using ultrasound images with the aid of a deep learning technique. ResearchGate. Published online December 16, 2024. doi:10.1007/s00500-024-10333-w

Summary of Open Improvement Edits

The 17 comments provided on our proposal raised valuable points that have been used to strengthen the proposal.  The majority of the comments came from clinical faculty and residents who manage the problem of PUL on a daily basis and believe this tool could be transformational on multiple levels.  We have tried to strengthen section 2 on How might AI help? in response to these helpful comments.

Summary of comments:

Comments supportive of the TRACE tool:

• Quality of care- The tool will improve the quality of patient care for this high-stakes condition. Though guidelines exist and have been validated in various populations, they are not always followed at ZSFG (or around the country for that matter) as one reader stated “I have found as a resident that these guidelines are not always followed, often due to providers simply not having enough time to refer to them every time, and instead going with their own personal preference when it comes to PUL management, and that leads to a lot of variation in practice and sometimes near misses. because providers simply don’t have the time to carefully map the specifics through the guideline (see attached) and instead rely on
• Administrative Burden- residents spend up to 3 hours daily following the floater list of PUL patients.
• Costs- The majority of patients with PUL are covered by Medicaid.  By streamlining follow up, costs will be saved while maintaining or improving quality.
• Patient-centered- Many patients are required to be followed from days to weeks with serial blood draws and ultrasounds to determine the location and viability of their pregnancy. This is disruptive and anxiety provoking with the potential for errors and missed ectopic pregnancies.  By creating a patient facing platform, communication can be automated and consolidated. Additionally, one reader commented on the value of being able to create language accessible messaging to our multi-lingual population served.  Often requires 10-15 min per call to secure interpreter and make certain message is understood

 In summary

On reader wrote, “AI can handle complex algorithmic decision-making that’s difficult for (tired) human brains to do quickly and it can complete important administrative tasks better than physicians (tracking, patient communication and reminders) making it a win for patients and physicians alike. If successful, TRACE would be a highly sought-after tool across the United States.” 

Comments to strengthen the proposal:

• One reader commented, how much of this could you do without AI? 

The fact is, all of it is currently done without AI which is suboptimal, time intensive, and risky.  We have added clarification to the description of the problem in section 1 to better justify the benefits of the TRACE tool. One of the readers, who is an ObGyn fellow, said it best- “PUL is a challenging clinical diagnosis because it carries significant uncertainty, is managed differently by different clinicians despite robust clinical guidelines on best practices, and is highly time and resource intensive for the trainees at our institution who do the patient-facing work of helping patients navigate the process of determining pregnancy location. This proposal has the potential to revolutionize care for patients facing an uncertain prognosis which can be confusing, scary and very burdensome, and also to revolutionize clinicians' work of managing PULs” 

• In response to the comment- It sounds as if developing or refining the initial prediction model is key -- driving the two other pieces. Can you expand a bit on how you might work with the validated tools you've mentioned? Has any preliminary work been done with UCSF data?

This is correct, validating and potentially modifying the M6 prediction model with UCSF data would be the first step. We have added additional clarification and references in the description of risk prediction models and the pathway we describe for development. Specifically, we will use the modified M6 model from the UK/Belgium and validate its performance with UCSF data.  The M6 model has been published and validated externally. We would need to work closely with UCSF data scientists for model validation using data points available from the UCSF EMR following the procedures described in the publications listed below. We will identify cases using the ICD 10 diagnosis code for PUL and extract data on BhCG, ultrasound results, and clinical characteristics (risk factors, symptoms) for our population. Initially, we expect to replicate the model using the initial visit and the first follow-up, however, going forward we hope to improve the predictive model utilizing information from all the follow-up visits so that it can inform decisions (such as when to return for the next visit/lab test/US) at each visit.

 Kyriacou C, Ledger A, Bobdiwala S, Ayim F, Kirk E, Abughazza O, Guha S, Vathanan V, Gould D, Timmerman D, Van Calster B, Bourne T; Collaborators. Updating M6 pregnancy of unknown location risk-prediction model including evaluation of clinical factors. Ultrasound Obstet Gynecol. 2024 Mar;63(3):408-418. doi: 10.1002/uog.27515. PMID: 37842861.

 Maheut C, Panjo H, Capmas P. Diagnostic accuracy validation study of the M6 model without initial serum progesterone (M6NP) in triage of pregnancy of unknown location. Eur J Obstet Gynecol Reprod Biol. 2024 May;296:360-365. doi: 10.1016/j.ejogrb.2024.03.010. Epub 2024 Mar 8. PMID: 38552504.

The UCSF Health problem

Inflammatory bowel disease (IBD), including Crohn's disease and ulcerative colitis, is a chronic inflammatory condition affecting 1.5 million people in the United States, with a significant hospitalization rate of 9.24 per 100 IBD patients annually [1]. Flares of the disease are a common cause of these hospitalizations. Optimal management of acute IBD flares necessitates timely surgical consultations, endoscopic evaluations, initiation of anticoagulant therapy, and possible surgical interventions [2]. However, adherence to these protocols is often compromised by heavy clinical workloads and oversight, leading to delays that diminish patient care quality, extend hospital stays, and increase healthcare costs [3].

UCSF, a tertiary medical center with a comprehensive IBD program, is committed to providing extensive care for patients with complex IBD cases necessitating hospitalization due to flares. Strict adherence to clinical protocols is vital for enhancing the quality of patient care, reducing the duration of hospital stays, and decreasing healthcare costs [3]. Despite these high standards, adherence to these protocols at UCSF, particularly at newly integrated sites such as Saint Francis Memorial Hospital and St. Mary's Medical Center, and during the absence of clinical fellows who assist the service, can fall short of expectations. This is largely due to the challenges posed by heavy clinical workloads and oversight.

How might AI help?

Our goal is to enhance the Advancing Patient-Centered Excellence (ApeX) electronic medical records (EHR) system at UCSF to improve monitoring and management of patients admitted with IBD flares. By integrating large language model (LLM) AI tools, such as VERSA at UCSF, both structured data (e.g., orders, lab values) and unstructured data (e.g., physician notes) collected during patient admissions can be effectively processed. This AI tool will actively monitor adherence to the established IBD management protocol and alert gastroenterology/IBD providers and primary team providers, such as hospitalists, by displaying a reminder window when they access a patient’s chart in the ApeX EHR system and being available as SmartPhrases available to providers when writing notes and signing off to the next provider in the hand-off in the APeX EHR system. This prompt aims to prevent delays in care by ensuring timely adherence to necessary clinical actions.

Key Areas Where AI Is Essential:

1. Imaging Interpretation: Determining whether imaging studies indicate colonic dilation requires parsing radiology reports. AI, particularly natural language processing (NLP) tools, can extract such findings from unstructured text, facilitating timely surgical consultations when necessary.​
2. Treatment Response Assessment: Evaluating a patient's response to intravenous glucocorticoids or biologic therapies involves synthesizing symptom descriptions, lab trends, and provider assessments documented in narrative form. AI models can integrate these data points to identify non-responders and prompt appropriate management adjustments.​
3. Abscess Monitoring: Assessing the resolution of a Crohn’s-related abscess post-antibiotic therapy and drainage requires tracking symptom improvement and imaging findings over time. AI can correlate these unstructured data elements to determine if further intervention is warranted, especially when the drainage is not adequate.​
4. Discharge Planning: Ensuring readiness for discharge encompasses verifying smoking cessation counseling, monitoring stool characteristics, and confirming nutritional tolerance—all typically noted in free-text clinical documentation. AI can collate this information to identify potential safety gaps prior to discharge, and prompt early initiation of discharge planning.
5. Admission Reasoning: Identifying admissions in which an IBD flare was not the primary reason for hospitalization, or during which a IBD flare developed, necessitates analyzing provider notes before the ICD codes available from the medical coding team. AI can detect such nuances and initiate IBD flare management monitoring ​as above at very early stage.
6. External Data Integration: Patients often receive care at multiple institutions, leading to fragmented records. AI can reconcile external documents, such as vaccination records or prior treatments, that are embedded in unstructured formats.
7. Reducing Unnecessary Alerts and Enhancing Clinical Workflow: Traditional electronic health record (EHR) systems often generate numerous alerts based solely on structured data, leading to alert fatigue among clinicians. AI can mitigate this by analyzing unstructured data to provide context-aware alerts. For the situation in the above, if a vaccination was administered at an out-of-network facility and documented only in clinical notes, AI can recognize this and prevent redundant alerts. By tailoring notifications to the physician's preferences and clinical context, AI enhances the usability of EHR systems, reducing the cognitive burden caused by irrelevant or excessive alerts.

How would an end-user find and use it?

Our objective is to enhance the Advancing Patient-Centered Excellence (ApeX) electronic medical records (EHR) system at UCSF, aiming to improve the monitoring and management of patients admitted with IBD flares. By incorporating large language model (LLM) AI tools, such as VERSA at UCSF, we can effectively process both structured data (e.g., orders, lab values) and unstructured data (e.g., physician notes) collected during patient admissions. This AI tool will actively monitor adherence to the established IBD management protocol. It will alert gastroenterology/IBD providers and primary team providers, such as hospitalists, by displaying a reminder window when they access a patient’s chart in the ApeX EHR system. Additionally, SmartPhrases will be made available to providers for use when writing notes and during the hand-off process to the next provider, ensuring seamless communication. These prompts are designed to prevent delays in care by ensuring timely adherence to necessary clinical actions.

Embed pictures of what the AI tool might look like

Figure 1 shows the reminder window for the IBD flare inpatient milestone not met: Failure to response to the first line therapy, but no secondary medical or surgery therapy started. The failure and is recognized by the AI tools, large language model, from the unstructured data, symptoms included in the provider and nurse notes. The deficiency of secondary medical or surgery therapy information is collected from structured data, medication orders, and notes, notes documenting surgical evaluation and procedure plan. A useful link is also attached to show the most updated guideline about the milestones for user education. And if the provider accepts it, it will open an order set directly for the convenience. But it does not force the provider to take any action, in case it is a falsely positive reminder. It works more as a reminder and allow the provider to check. The “Decline” button will allow the user to leave a comment to help with the tool.

 Figure 2 depicts a SmartPhrase that shows the current IBD flare inpatient milestone meeting status. It will contain following menus to allow the provider to select if necessary and allow the provider to do any edits. The SmartPhrase will also work in the Handoff part for the purpose of signing off and providing information for other teams, like the night on call shift or other consult teams. 

What are the risks of AI errors?

AI tools may occasionally misinterpret data or generate errors, leading to false positives or false negatives. For instance, a milestone may not be met, yet the AI might incorrectly indicate that it has been met (false positive). However, this is unlikely to cause additional harm, since a provider who would have recognized the unmet milestone independently would not be misled into overlooking it simply because the AI suggested otherwise. The only scenario in which a milestone would still be missed is if both the provider and the AI tool fail to recognize it—essentially the same outcome as not using the AI tool at all. Conversely, the AI might fail to recognize that a milestone has been met, resulting in a false negative. However, these errors can often be corrected through provider review. Given that these milestones are generally straightforward for providers to verify, but may be overlooked due to heavy clinical workloads, the AI tool primarily functions as a reminder rather than a decision-maker.

How will we measure success?

The implementation and evaluation of the AI tool for IBD flare management necessitate a strategic approach to measure its effectiveness and minimize disruption of clinical workflow and maximize provider adoption and satisfaction. The evaluation process will be structured into two primary components:

1. Evaluate the Effectiveness of Protocol Adherence Improvement: Prospective cohort study: Patients admitted with IBD flares before applying this tool and those after applying this tool and using the tool by their providers will be assigned to two different groups: This approach allows us to directly compare outcomes between the groups.
   • Primary Outcome—Protocol Adherence: Adherence will be quantitatively assessed using a specifically designed scoring system that evaluates how closely care teams follow established management protocols.
   • Secondary Outcomes—Health Costs and Hospital Stay Duration: We will measure the economic impact by considering potential costs associated with AI-related errors, such as false negatives that could lead to unnecessary workups, and potential savings by reducing the complication due to delay of care. Additionally, the length of hospital stays will be tracked to assess any efficiencies gained through improved management adherence.
2. Evaluate Provider Satisfaction:
   • User Feedback and Scoring System: The acceptance and satisfaction of inpatient providers using the AI tool are critical for its long-term success. We will collect Net Promoter Scores (NPS) to quantify user satisfaction. Additionally, we will gather detailed feedback for further improvements, capturing both qualitative and quantitative aspects of user experience.

Existing APeX Data Metrics: Protocol Adherence, Hospital Stay Duration.

Additional Ideal Metrics: Health Costs, User Feedback, and Scoring.

Describe our qualifications and commitment

This project is spearheaded by Yuntao Zou, MD, a seasoned hospitalist with extensive experience in inpatient medical management. Dr. Zou is also an accomplished AI researcher, with a clinical and research focus on leveraging AI tools, such as Large Language Models (LLMs), to enhance healthcare delivery and clinical decision-making processes. I am responsible for designing and overseeing the entire project process. If selected, I will devote at least 10% or as required effort for 1 year to ensure the success of this proposal.

Vivek Rudrapatna, MD, PhD, serves as the co-lead for this initiative, with a specific focus on enhancing the IBD management protocol and developing the AI tool. As a physician-scientist and specialist in inflammatory bowel disease, Dr. Rudrapatna brings a wealth of experience in IBD management. He also leads a research group dedicated to developing methods for analyzing healthcare data, aiming to enhance clinical decision-making processes.

Reference:

1.           Buie, M.J., S. Coward, A.A. Shaheen, J. Holroyd-Leduc, L. Hracs, C. Ma, et al., Hospitalization Rates for Inflammatory Bowel Disease Are Decreasing Over Time: A Population-based Cohort Study. Inflamm Bowel Dis, 2023. 29(10): p. 1536-1545.

2.           Lewin, S. and F.S. Velayos, Day-by-Day Management of the Inpatient With Moderate to Severe Inflammatory Bowel Disease. Gastroenterol Hepatol (N Y), 2020. 16(9): p. 449-457.

3.           Burisch, J., M. Zhao, S. Odes, P. De Cruz, S. Vermeire, C.N. Bernstein, et al., The cost of inflammatory bowel disease in high-income settings: a Lancet Gastroenterology & Hepatology Commission. Lancet Gastroenterol Hepatol, 2023. 8(5): p. 458-492.

 Summary of Open Improvement Edits

• Changed Figure 1 and its explanation to an example more related to the requirement of AI tools.
• Thanks to Dr. Pletcher reminding! Key features requiring AI tools added in the "How might AI help?" part.
• Thanks to Dr. Xue reminding! Added the influence of the falsely positive result (missing the milestones not met) in the "What are the risks of AI errors?" part.

The UCSF Health problem

Plain knee radiographs are a cornerstone in evaluating osteoarthritis (OA), and at UCSF, nearly all such studies include Kellgren-Lawrence (KL) grading to assess severity. While essential, this grading task is repetitive, time-consuming, and inherently subjective—especially in borderline cases^1,2. In fact, studies show only moderate inter-reader reliability in KL grading, creating inconsistencies in diagnosis and downstream care decisions^3,4.

KL grading adds to radiologist workload⁵, reducing time available for complex cases and clinical consults. Despite its clinical importance, few innovations have addressed this burden at scale. Although machine learning methods for KL grading have shown promise in research settings, these tools have yet to be integrated meaningfully into clinical workflows.

Our proposed solution targets a high-volume, repetitive task to reduce radiologist burden, improve grading consistency, and support radiology education. Our goal is not to replace clinical judgment, but to reinforce it with consistent, reproducible assessments of OA severity.

How might AI help?

We propose integrating an automated AI model for KL grading directly into the radiology workflow at UCSF. Our model has been trained on thousands of local clinical knee radiographs and has demonstrated strong agreement with expert radiologist grading. The tool uses a deep learning pipeline to automatically detect knee joints and assign KL grades (0 to 4, or total knee replacement). This structured information would be embedded directly into the radiology report and supported with visual overlays to guide interpretation. We hypothesize that this system will improve inter-reader reliability, assist radiologists in ambiguous cases, and reduce the clinical workload associated with routine KL scoring.

Automating KL grading using deep learning has the potential to significantly enhance clinical practice by improving grading consistency and reducing the workload of radiologists. Despite promising research, automatic KL grading has not yet been integrated into clinical workflows. Our proposed study would demonstrate the feasibility and effectiveness of such integration, showing that an automated system can streamline workflow, reduce reporting burden, and provide decision support for ambiguous cases.

Key beneficiaries of this tool are:

- Radiologists: With KL grading offloaded to the algorithm, radiologists can devote more time to more complex interpretations and high-value consults. Structured outputs integrated into PowerScribe reduce cognitive load and speed reporting.

- Patients: Consistent and reproducible KL scores reduce the risk of diagnostic variability. More available radiologist time means better patient communication and faster turnaround.

- Trainees: The system’s visual outputs and attention maps can serve as an educational aid for residents and fellows learning musculoskeletal imaging, helping them learn to identify KL grades more reliably and confidently.

By deploying this AI system at the point of care, we can support more efficient workflows, better diagnostic consistency, and enhanced trainee learning.

As a proof-of-concept for automatic KL grading for clinical knee OA scoring at UCSF, we have developed a deep learning pipeline which effectively detects the knee joint on clinical radiographs and automatically classifies OA severity. In this project we trained two models: an object detection model to first perform cropping around the knee joint, and a classification model to identify KL grade (0-4) or presence of total knee replacement (TKR).

For object detection, 814 unilateral AP knee radiographs from the Osteoarthritis Initiative (OAI)⁹ were used to train image cropping around the knee joint. A You Only Look Once (YOLO, version 8)¹⁰ object detection model was trained on OAI radiographs. An 80/10/10 split was applied for training, validation, and test sets. Mean intersection-over-union values between predicted and ground truth bounding boxes for the test set, training set, and validation set were 0.8947, 0.8845, and 0.8635, respectively. Applying object detection to UCSF clinical knee radiographs, at least one knee joint was detected in 94.73% (n = 10,842) radiographs and cropped.

For classification, 9,166 anonymized clinical knee radiographs were acquired from UCSF PACS AIR¹¹ for training, validation, and prediction. 4,978 bilateral and 4,188 unilateral radiographs. KL labels were extracted from corresponding UCSF radiology reports were extracted using regular expression and used to run a pretrained EfficientNet-B7¹² classification model on an 80/10/10 training/validation/test data split of cropped and KL-labeledUCSF knee radiographs. Weighted Cohen's Kappa showed substantial agreement (0.74 for validation set and 0.76 for test set).

How would an end-user find and use it?

The AI system will be integrated with clinical radiology tools Visage Picture Archiving and Communication System (PACS) and Nuance PowerScribe reporting software. Radiological images will be routed to the KL grading software based on a set of filters, such as modality and body part. The models will detect left and right knee joints and generate numerical KL scores of osteoarthritis severity: 0 – No OA, 1 – Doubtful OA, 2 – Mild OA, 3 – Moderate OA, 4 – Severe OA, 5 – hardware, such as artificial joints from total knee replacement.

When a knee radiograph is opened for interpretation, suggested grades, along with brief descriptors of severity (e.g., "mild OA"), will be pre-populated into the PowerScribe radiology report template. Radiologists can edit or accept the AI-suggested grades. Additionally, an annotated image with overlaid KL scores and saliency maps (indicating the most relevant areas of the image used for classification) will be available for review in the Visage PACS viewer software. Trainees can use these visualizations to better understand the features driving each grade. This process occurs passively and seamlessly, saving time and offering real-time decision support. See example mock-up of the end-user interface below:

What are the risks of AI errors?

There are two primary categories of errors:

Failure to detect the knee joint: This would result in the system being unable to suggest a KL grade. However, our preliminary data show that the model successfully detects the knee joint in over 95% of cases, making this scenario unlikely. When it does occur, the fallback is manual grading by the radiologist, as is currently done.

Misclassification of osteoarthritis severity: This includes both false negatives (e.g., missed severe OA), which may lead to underestimation of disease severity and undertreatment, and false positives (e.g., overgrading of mild cases), which may lead to unnecessary further evaluation or intervention.

The rate of these failures will be measured comparing the record of model output to the final radiology reports. Our preliminary data showed good agreement between generated KL grades and ground truth values, assigned by radiologists. To mitigate these risks:

• AI-generated grades will never bypass radiologist review. Only radiologist-approved grades will be included in the final report.
• We will conduct a quality assurance study comparing AI grades to independent radiologist adjudication.
• Continuous monitoring of agreement between AI predictions and final report grades will be conducted, and discrepancies will be reviewed to guide system refinement.

How will we measure success?

Initial evaluation: In a randomized adjudication study, two radiologists will independently compare prior clinical KL grades and AI-generated grades. We will measure inter-rater and model agreement using Cohen’s Kappa, with success defined as the model matching or exceeding inter-radiologist agreement.

Clinical impact: We will evaluate how frequently AI-generated grades are accepted without edits, and whether AI usage improves intra- and inter-reader consistency across reports. We will assess radiologist-reported satisfaction with the tool and trainee confidence and accuracy in KL grading via structured surveys and explore whether AI integration leads to measurable efficiency gains, including reduced interpretation time per case and an increase in the number of radiographs read per day.

Continuous Evaluation: We will implement a dashboard for continuous monitoring of the model performance by recording the generated grades and comparing them with the final grades entered into the reports. Any significant decrease in the rate of agreement will be followed up with model fine-tuning with additional data.

Describe your qualifications and commitment

This project is led by Dr. Yuntong (Lorin) Ma, MD, Assistant Professor of Radiology at UCSF, and Eugene Ozhinsky, PhD, Associate Professor of Radiology at UCSF.

Dr. Ma’s work focuses on developing deep learning solutions for musculoskeletal imaging, including automated classification of osteoarthritis severity and diagnosis of inflammatory arthropathy. Her research emphasizes practical clinical integration, and she actively contributes to shaping standards for the responsible deployment of imaging AI. Her ongoing research focuses on advancing the clinical implementation of AI tools in ways that are practical and relevant to patient care. If selected, she will commit 10% effort for at least 1 year towards this project to ensure its success.

Dr. Ozhinsky’s research focuses on applying advanced image acquisition and machine learning techniques to improve diagnosis, predict disease progression, and guide therapy—particularly in musculoskeletal conditions. He has developed AI models for tasks such as hip fracture detection, automated OA grading, and MRI protocol optimization. His long-term goal is to translate these novel techniques into routine clinical care so that they result in meaningful improvements in patient outcomes.

Drs. Ma and Ozhinsky will oversee a multidisciplinary team of scientists and engineers in close collaboration with the UCSF Center for Intelligent Imaging. The team meets weekly to review progress, troubleshoot challenges, and plan next steps. Regular engagement with key clinical stakeholders will guide implementation, including the Radiology AI Governance Committee, UCSF Health AI, and AER leadership.

References

1.         Kohn MD, Sassoon AA, Fernando ND. Classifications in Brief: Kellgren-Lawrence Classification of Osteoarthritis. Clin Orthop Relat Res. 2016;474(8):1886-1893. doi:10.1007/s11999-016-4732-4

2.         Braun HJ, Gold GE. Diagnosis of osteoarthritis: imaging. Bone. 2012;51(2):278-288. doi:10.1016/j.bone.2011.11.019

3.         Wright RW, MARS Group. Osteoarthritis Classification Scales: Interobserver Reliability and Arthroscopic Correlation. J Bone Joint Surg Am. 2014;96(14):1145-1151. doi:10.2106/JBJS.M.00929

4.         Köse Ö, Acar B, Çay F, Yilmaz B, Güler F, Yüksel HY. Inter- and Intraobserver Reliabilities of Four Different Radiographic Grading Scales of Osteoarthritis of the Knee Joint. The Journal of Knee Surgery. 2017;31:247-253. doi:10.1055/s-0037-1602249

5.         Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S. Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach. Sci Rep. 2018;8:1727. doi:10.1038/s41598-018-20132-7

6.         Lee LS, Chan PK, Wen C, et al. Artificial intelligence in diagnosis of knee osteoarthritis and prediction of arthroplasty outcomes: a review. Arthroplasty. 2022;4:16. doi:10.1186/s42836-022-00118-7

7.         Swiecicki A, Li N, O’Donnell J, et al. Deep learning-based algorithm for assessment of knee osteoarthritis severity in radiographs matches performance of radiologists. Computers in Biology and Medicine. 2021;133:104334. doi:10.1016/j.compbiomed.2021.104334

8.         Norman B, Pedoia V, Noworolski A, Link TM, Majumdar S. Applying Densely Connected Convolutional Neural Networks for Staging Osteoarthritis Severity from Plain Radiographs. J Digit Imaging. 2019;32(3):471-477. doi:10.1007/s10278-018-0098-3

9.         NIMH Data Archive - OAI. Accessed November 4, 2024. https://nda.nih.gov/oai

10.       Redmon J, Divvala S, Girshick R, Farhadi A. You Only Look Once: Unified, Real-Time Object Detection. Published online May 9, 2016. doi:10.48550/arXiv.1506.02640

11.       AIR Overview. UCSF Radiology. May 29, 2018. Accessed March 20, 2025. https://radiology.ucsf.edu/research/core-services/PACS-air

12.       Tan M, Le QV. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Published online September 11, 2020. doi:10.48550/arXiv.1905.11946

The UCSF Health Problem

Despite the availability of clinical practice guidelines from the American Association for the Study of Liver Diseases (AASLD) and the American Gastroenterological Association (AGA), adherence to recommended quality measures for patients with cirrhosis remains suboptimal.  This gap leads to a high burden of readmissions and inpatient mortality bore by patients with cirrhosis – avoidable healthcare costs.  Cirrhosis is a dynamic condition that affects multiple organ systems outside of the liver, e.g. brain (encephalopathy), hematology (cytopenias), renal (hepatorenal syndrome), cardiology (cirrhosis related cardiomyopathy), and infectious diseases (spontaneous bacterial peritonitis); this complexity is reflected in the electronic health record, which typically includes relevant clinical data across multiple parts of the patient record.  Additionally, busy clinicians may find it challenging to quickly reference and integrate multiple clinical guidelines.  There is an urgent need for a streamlined solution that ensures guideline-concordant care tailored to each patient’s specific clinical profile and decompensations.

How Might AI Help?

We have previously constructed a liver disease specific Large Language Model (LLM) called “LiVersa” based on integration of AASLD clinical practice guidelines via Retrieval Augmented Generation (RAG-LLM).  LiVersa is the first clinical assistant in the UCSF Versa platform and is currently available for select clinicians in hepatology and liver transplantation (PMID 38451962, PMID 38935858).  We have also constructed “CirrhosisRx,” which is a rule-based (non-AI) clinical decision support system for guideline adherent inpatient cirrhosis care, built on the EngageRx platform using SMART-on-FHIR.  CirrhosisRx is currently deployed in a pragmatic randomized clinical trial (NCT05967273, PMID 38407255) at UCSF Medical Center.  In this proposal, we plan to link the two technologies, specifically integrating LiVersa into CirrhosisRx via Fast Healthcare Interoperability Resources (FHIR) application progressing interface (API) calls within the APeX EPIC EHR system, to provide a patient-personalized dynamic information retrieval system to provide guideline-based clinical recommendations to end-users.

The integrated LiVersa-CirrhosisRx system would provide the following advantages:

• Both Rule-Based and Generative Hybrid Recommendations:  Integration of LiVersa RAG-LLM into CirrhosisRx will allow providers to query and access up-to-date AASLD clinical guidelines without navigating outside of the EHR.
• Personalized Medicine: The proposed application would automatically incorporate relevant patient-specific data, enabling real-time, personalized recommendations specific to the patient and clinical scenario (“Right Patient, Right Time”).
• Streamlined Workflow: Embedding the LLM in a user-friendly CDS interface eliminates the need to toggle between multiple resources, reducing cognitive load and time constraints on clinicians.

How Would an End-User Find and Use It?

The combined LiVersa-CirrhosisRx application will be deployed within the existing “CirrhosisRx” tab that is enabled for select EPIC user contexts under the ongoing pragmatic randomized clinical trial.  See screenshot of CirrhosisRx:

In the LiVersa-CirrhosisRx integration, we will use the FHIR data connected with LiVersa to generate a summary of the patient with displayed data.  A customized free-text box will be built into the CirrhosisRx application that allows the end user to ask open-ended questions regarding the management of the patient.  The embedded LiVersa RAG-LLM will return answers that reference the latest AASLD guidelines while reflecting the patient’s unique clinical situation.  Future expansions could include automated pending of orders and order sets consistent with LiVera’s recommendations.

Example of AI Output

What Are the Risks of AI Errors?

There several potential risks arise when introducing an LLM-based solution into clinical workflows: 1. Hallucinations or Misinformation, 2. Clinician bias due to overreliance on AI, 3. Biases in recommendations, 4. Clinical scenarios un-accounted for by the RAG-LLM model.  To mitigate these risks, we propose a robust pilot testing phase embedded within the existing CirrhosisRx trial.  We will record both clinical outcomes, defined as adherence to AASLD/AGA practice guidelines for inpatient cirrhosis care, and implementation outcomes through structured usability assessments and semi-structured interviews.  We have been developing LiVersa for a use case of hepatology e-consultations and have a 12-question survey developed to evaluate its effectiveness versus human-written consultation recommendations in 54 previous e-consult cases.  The preliminary data from our analyses have demonstrated that the e-consultation drafts produced by LiVersa were helpful 71% of the time with a 4% potential harm rate.  Given that all clinical actions taken resulting from the LiVersa-CirrhosisRx application would have to be confirmed/finalized by a human clinician, there is an integral “human-in-the-loop” mechanism for this proposal.

How Will We Measure Success?

We will track both clinical outcomes and implementation outcomes to determine the effectiveness of our LLM-enhanced CDS tool:

• Clinical Outcomes:

• Guideline adherence - Measure adherence to five AASLD/AGA guideline recommendations, comparing EHR-based metrics before and after the tool’s introduction.
• Rates of Hospital Readmission: Evaluate changes in readmissions in patients with cirrhosis taking place with 90 days.
• Mortality and Morbidity Trends: Evaluate changes in inpatient mortality rates for patients with cirrhosis.

• Implementation Outcomes:

• Time Savings: Track time spent in chart review and decision-making tasks.
• Frequency of CDS Usage: Evaluate how often providers use LiVersa-CirrhosisRx interface in the context of how many potential encounters where it could be used.
• Clinician Surveys: Gather feedback on trust in AI recommendations, ease of use, and perceived impact on practice.

• Equity and Bias Analysis

• Demographic Subgroup Performance: Examine whether the AI recommendations remain consistent across different patient populations with varied demographic backgrounds.

Describe Your Qualifications and Commitment

• This project is led by Dr. Jin Ge, MD, MBA.  Dr. Ge is a transplant hepatologist and a data science and AI researcher within the Division of Gastroenterology and DoC-IT.  He serves as the Director of Clinical AI for the Division of Gastroenterology.  He has experiences in developing, testing, and deploying digital technologies for liver disease care and is the principal investigator for the CirrhosisRx pragmatic randomized controlled trial and LiVersa liver-disease specific LLM.  He has worked closely with both APeX-Enabled Research and the AI Tiger Team.  If selected, he will commit at least 10% effort for 1 year towards this project to ensure its success.
• Dr. Valy Fontil is a former faculty member at UCSF and currently serves as the Director of Research at Family Health Centers at NYU Langone Health.  He is a primary care physician, health services researcher, and digital health entrepreneur who specializes in innovations for high-risk, low-income populations.  He is the innovator of the EngageRx platform, on which CirrhosisRx is based, and has expertise in building digital health interventions in the EHR.  He will serve as an advisor to this effort.

Section 1: The UCSF Health Problem

At UCSF and other leading academic medical centers, the referral intake and triage process for new patients is strained, leading to long delays and high rates of incomplete referrals. A review of more than 100,000 referral scheduling attempts showed wide variability in wait times—from 8 to 73 days—with an average of 22 days [1].  Additionally, ~50% of referrals are never completed, largely due to operational bottlenecks and limited triage capacity [2,3].

These delays are not just administrative—they directly affect patient outcomes. In a consecutive cohort study of 648 patients with squamous cell carcinoma, the majority of patients developed significant signs of tumor progression within a wait time of 4 weeks [4]. For time-sensitive conditions like lung, kidney, and pancreatic cancer, each week of delayed treatment increases mortality risk by 1.2% to 3.2% [5]. In early-stage cases, delays in cancer care can raise 5-year and 10-year mortality rates by as much as 47.6% and 72.8%, respectively [5]. 

A key challenge lies in how referrals are routed and scheduled. At UCSF, the status quo workflow is labor-intensive. Referral documents arrive from outside providers as faxes or PDFs that must be manually reviewed. Currently, this triage process falls to practice coordinators, who work hard to manage a high volume of incoming cases while juggling many responsibilities. However, they typically do not have clinical training, and determining the right subspecialist and urgency level for complex cancer cases often requires input from nurse navigators or physicians. Even when marked urgent, referrals can take days to reach the right destination—such as in head and neck oncology, where the average triage time is over 4 days.

To improve access, UCSF has introduced a patient self-scheduling portal for some specialties. While this is a step in the right direction, the current system is limited in scope and with various levels of oversight. It relies on a series of 3 generic questions and lacks integration with predictive models or clinical context. As a result, it does not capture the complexity of real-world triage, which can lead to patients being misrouted. Specialty clinics may also be scheduled suboptimally, with patients who may have benefited from additional work-up (e.g., APP visit) or additional medical record reconciliation. These challenges lead to financial losses, reduced market competitiveness, and prevent subspecialists from focusing on high-value, "top-of-license" care.

While past efforts—like hiring additional staff or sending reminders—have helped incrementally, they don’t address the core issue: patients need better tools to navigate the referral process, and staff need support to triage more efficiently.

Our solution is designed to bridge this gap by embedding intelligent triage and decision support into the patient self-scheduling experience—helping patients land in the right clinic faster while reducing the load on care teams and improving access to timely treatment. Importantly, this project also aligns with 1 of the 4 UCSF Ambulatory Services Health IT Portfolio Initiatives for FY2025. 

Section 2: How might AI help? 

We want to enable safe, accurate, and most notably, intelligent self-scheduling for new specialty patients (Figure 1). Our solution consists of an AI-powered triage algorithm developed through collaboration with IIAM Corporation and taps into their existing software platform’s advanced capabilities. Once deployed, our algorithm will assist with the patient self-scheduling workflow, thereby reducing dependence on manual triage and improving the timeliness of care. 

Figure 1. UCSF Self-Scheduling Workflow and Proposed Intervention. The top row demonstrates UCSF’s current online self-scheduling portal. Patients are prompted to answer 3 in-descript questions about whether or not they have cancer, but many patients are unsure of their diagnosis. In the bottom row, we propose embedding IIAM’s algorithm into the online patient self-scheduling portal and using the associated referral documents to showcase provider names and appointment availability in accordance with the suspected etiology and urgency, maximizing clinical optimization (“top of license work”) and resource utilization. 

Our algorithm accepts previous medical records (e.g., clinical notes, radiology/pathology reports) as input data, including external documents. To ensure a smooth workflow, our algorithm will be flexible enough to accept multiple file formats including PDFs, images, and plain text files. The uploaded information is then reviewed by machine learning models to identify the patient’s current medical need and output the best matching subspecialty physician(s). For instance, a patient with a suspicious thyroid nodule will be matched with the appointment times of head & neck surgeons specializing in thyroid surgery. This approach will increase the rate of correct patient-physician matching and one-contact resolution, reduce time to treatment, increase surgical conversion rates in clinic, and maximize “top of the license work” among clinical providers.

The IIAM software platform has achieved a referral accuracy rate nearly 30% higher than labor-intensive, personnel-driven status-quo workflows (90% vs 60%) -- even with limited patient information (90% accuracy rates with 30-70% patient records missing). This clinical effectiveness was confirmed using patient data from the two largest national cancer databases (SEER, NCDB) and three tertiary healthcare centers (UCSF, Hopkins, MGB). At UCSF, our product has undergone both retrospective and prospective validation. Our algorithm was 100% accurate in identifying malignancy. Our team hopes to utilize the UCSF AI Pilots program to develop a solution that is more tailored to the institution’s specific self-sceduling needs and refine algorithm performance via training on UCSF clinical data.

Section 3: How would an end-user find and use it? 

The AI tool will be integrated into UCSF’s existing online scheduling system, where it will prompt new patients seeking a specialty visit to upload relevant medical documents—such as referral letters, imaging results, pathology reports, or lab work. 

Once documents are submitted, IIAM’s AI analyzes the content to understand the underlying condition, clinical urgency, and appropriate subspecialty. Within seconds, the patient receives a tailored list of providers with real-time appointment availability, ranked by clinical fit and urgency, based on the content of their documents (Figure 2). The patient then selects from the AI-filtered list of appointment options, enabling them to self-schedule with an appropriate provider without waiting for manual triage, provider verification, or callbacks. If the AI determines that a more urgent evaluation is needed, it may recommend an earlier visit slot or flag the case for real-time escalation to nurse triage.

The system is designed to require minimal effort from the patient while maximizing the value of any existing work-up they have completed. Importantly, the AI acts as a behind-the-scenes assistant—not a gatekeeper. Patients retain the ability to view other available providers or request help if needed. For internal users (e.g., nurse navigators or access center staff), a clinical summary of the AI’s triage decision can also be displayed to assist in complex case management.

To minimize any errors, we intend to retain a human-in-the-loop during the triage and prior to scheduling for the first three months of the live pilot. If success metrics and accuracy rates remain high, the team will discuss and consider gradually scaling back the human-in-the-loop involvement. The algorithm provides a confidence score with every referral, and any referral that is associated with a low confidence score will automatically be flagged for human review. 

Section 4: Embed a picture of what the AI tool might look like. 

Figure 2. Proposed UCSF Self-Scheduling Workflow. Based on the patient’s pre-existing work-up and documents, the UCSF patient self-scheduling portal will provide the patient with an AI-filtered list of appointment options with subspecialists that treat the patient’s existing condition. 

Figure 3. UCSF Self-Scheduling User Interface. A picture of the patient user interface with the AI-filtered list of appointment options with and appropriate subspecialists

Section 5: What are the risks of AI errors? 

False negatives—cases where urgent conditions are missed—can lead to harmful delays in care. In contrast, false positives may cause patients to be seen earlier than necessary, potentially burdening provider schedules. To mitigate this, the algorithm is intentionally designed to favor false positives over false negatives when evaluating the urgency of a patient’s chief complaint. This approach was developed in close collaboration with oncology providers, based on the shared principle that it is preferable to evaluate a benign lesion too early than to delay care for a potential cancer patient.

Encouragingly, the algorithm has demonstrated a high level of accuracy across leading healthcare systems (Figure 4). It significantly outperformed the traditional personnel-based call center at Johns Hopkins, achieving 87% accuracy using a random forest model, compared to 60% under current workflows. At UCSF, the algorithm achieved approximately 90% accuracy when benchmarked against physician assessments and pathology-confirmed diagnoses (Figure 5). These real-world results suggest a reliable and scalable solution for streamlining referrals and improving timely access to specialty care.

Figure 4. IIAM performance at both UCSF and JHMI. IIAM performance on incoming referrals during 2024 at both Johns Hopkins and UCSF. At JHMI, status-quo referral workflows involve a centralized call center and EPIC-based random forest tree algorithms, with a baseline performance of 60%. The physician’s assessment/plan and any surgical pathology reports served as the ground truth.

Figure 5. Confusion matrix for IIAM ML algorithm and UCSF Head and Neck Pathologies. Pathologies were determined from physician assessment and any related pathology results during a patient visit. Of the pathologies, 96% of non-endocrine neoplasm pathologies matched algorithm pathology predictions; 85%  of benign lesion pathologies matched algorithm pathology predictions; 96% (26/27) of thyroid pathologies matched algorithm pathology predictions; 100% of parathyroid pathologies matched algorithm pathology predictions; and 100% of salivary gland pathologies matched algorithm pathology predictions. 

Section 6. How will we measure success?

Measurements using data that is already being collected in APeX: 

• New patient referral volume
• Number of patients scheduled per month via the self-scheduling portal
• Time to triage
• Time to schedule 
• Time to treatment

Other measurements you might ideally have to evaluate the success of the AI 

• Percentage of appropriately scheduled referrals (etiology and urgency)
• Satisfaction scores (patients, providers, practice coordinators)
• Percentage of incoming referrals scheduled (UCSF H&N baseline: 62%)
• Surgical conversion rate aka optimal clinical utilization (e.g., cancer patients that are non-surgical candidates see medical oncology first; non-biopsied patients see ENT first; benign lesion or incomplete work-up see an APP first)

Section 7: Describe your qualifications and commitment: 

Our team combines deep clinical expertise with a proven track record of applying AI/ML solutions to real-world healthcare challenges. With a shared commitment to improving patient access and outcomes—particularly in cancer care—we are uniquely positioned to lead this initiative.

Katherine Wai, MD is a head and neck cancer surgeon-scientist in the UCSF Department of Otolaryngology–Head and Neck Surgery. She has published peer-reviewed research on the use of AI/ML to improve triage and referral processes for cancer patients, reflecting her dedication to innovation in care delivery. Dr. Wai is the principal investigator for UCSF’s pilot studies involving IIAM technology and will ensure the project meets and exceeds national quality improvement benchmarks.

Patrick Ha, MD is the Medical Director of UCSF Mission Bay Adult Services and holds the Irwin Mark Jacobs and Joan Klein Jacobs Distinguished Professorship in Head and Neck Surgery. An international expert in head and neck cancer research and outcomes, Dr. Ha brings deep expertise in NCCN and AJCC cancer guidelines. He will oversee the integration of AI-driven solutions into UCSF’s self-scheduling workflows and ensure alignment with UCSF Cancer Center's patient access and strategic goals.

Nicole Jiam, MD is the Director of the UCSF Otolaryngology Innovation Center and Chief Executive Officer of IIAM Health. A clinical informaticist with experience collaborating across leading academic health systems—including Mass Eye and Ear and Johns Hopkins—Dr. Jiam has authored multiple peer-reviewed publications on AI/ML in healthcare and holds patents from both institutions. She brings a rare blend of clinical, academic, and entrepreneurial experience, having served on advisory boards for health tech companies nationwide and as a former fellow with digital health-focused VC firms. Dr. Jiam will actively guide product development and participate in regular progress reviews with UCSF’s AER team.

Together, Drs. Wai, Ha, and Jiam lead a cross-disciplinary effort grounded in clinical excellence and operational insight. With active involvement from UCSF leadership, their work reflects a sustained commitment to transforming cancer care access through scalable, AI-powered solutions.

1. The UCSF Health Problem

Patients at UCSF Health who receive immunosuppressive medications (e.g., disease modifying drugs (DMARDs) like methotrexate or biologics) require frequent laboratory monitoring to detect potential toxicities such as liver enzyme elevation, cytopenias, elevated blood pressure, organ dysfunction, or opportunistic infections. Although guidelines recommend regular laboratory monitoring, these processes rely on manual workflows during clinic visits and are unfortunately ad hoc in nature, leading to significant patient safety risks. Further complicating monitoring, tests may be performed in outside laboratories and therefore only captured in clinical media in pdf form (or discussed during the visit) within the EHR. National studies done by our team also demonstrate that critical safety monitoring for immunosuppressive medications frequently is not done or delayed.¹² As a result, patients may face delayed therapy adjustments and unnecessary risk of medication-induced harm, with crucial lab values or infection screenings failing to surface in real time. Existing methods—such as manual spreadsheets or nurse phone reminders, are labor-intensive and difficult to scale. Operational leadership in rheumatology at UCSF has identified development of improved lab monitoring systems as a high-priority target for improvement, noting that automated, comprehensive approaches could significantly increase efficiency, reduce legal risks to the institution, and most importantly, enhance patient safety.

2. How Might AI Help?

We propose an AI tool to automatically monitor drug safety gaps. The tool would identify patients overdue for recommended labs based on medications, laboratory data, outside lab tests contained in PDF documents, and labs that were entered by clinicians in notes. We envision this tool as a system with a dashboard as the frontend; the dashboard would streamline tracking of required safety tests, while minimizing false alarms regarding missed screenings, and reducing the need for manual chart review, both increasing safety and decreasing administrative burdens. Specifically:

• The tool will select patients who are currently receiving a medication of interest that requires periodic testing, including oral DMARDs and biologic drugs. Required tests and their limit dates will be kept in a table the tool has access to, such as a csv file, which is updated daily.
• The tool will then retrieve all relevant structured EHR data (laboratory results and other relevant screenings) for patients and flag those with significant delays in monitoring. By using existing branching algorithms and prior experience with Clarity pulls, we could quickly implement this real-time data pull using LOINC codes. For example, methotrexate requires laboratory monitoring every 3 months; the tool will flag all patients who do not have evidence of monitoring in the last 5 months. This grace period will identify patients who are clearly outside guideline recommended screening windows.
• For flagged patients, the tool will then leverage Versa through the API, screening notes and patient portal messages from recent visits for mention of labs and tests of interest that may have been performed externally. For PDF documents in clinical media, the tool will use an OCR algorithm (or Python extraction if digital) followed by Versa. If information is found, the tool will update its data, removing the flag, indicating that the patient did receive their periodic testing in time, and presenting lab results in the dashboard. This will allow us to avoid Versa running unnecessarily. We have prior experience in prompt design and in-context learning methods using Versa for information extraction, even when the information is not explicitly available but requires model reasoning, and will use this expertise here.³
• The AI tool will raise a warning for all patients that remain flagged (i.e. no testing data found in either structured data, notes, or clinical media PDF files).
• The loop will be repeated as needed to ensure patients continue to receive their safety testing. The loop will also include criteria for exclusion (for example, the medication is no longer prescribed, or the patient is no longer seen at the clinic, with functionality for staff to update this latter component in the platform). We will explore how often to re-send alerts that are not acted upon, and re-run the loops, through qualitative interviews with the care team.

3. How Would an End-User Find and Use It?

The AI tool would run in the background and require no UI. Interaction is limited to: (1) MAs receiving alerts, who will then communicate with other members of the care team and patients as needed, and (2) loop results saved in a dashboard the care team can consult. The dashboard, which will be iteratively designed with clinical staff, may also include search functionality to customize checks for individual patients (e.g., a patient that has a comorbidity that requires more frequent monitoring) as well as have a feedback option, so that the study team can monitor problems as they come up. The figure below presents a diagram of our proposed model.

4. What Are the Risks of AI Errors?

• False Positives: The system might raise a false alarm by missing documentation of a lab that has already been completed and documented in the EHR. To mitigate this risk, we will do extensive testing with human annotators and staff during pilot testing specifically asking if they have observed any false positives, and if so, we will retrain/adjust the model. We will ensure this risk is minimal by conducting chart review and model evaluation during an initial proof of concept phase. Once fully implemented, model maintenance will include regularly scheduled validation to ensure the model continues to function properly, preventing model drift. Feedback collection will be integrated in the dashboard.
• False Negatives: The system fails to detect a truly missed test. The feedback system designed above will also detect and mitigate this risk if present. We will also assess the prevalence of false negatives during the proof-of-concept phase and regularly though implementation.
• Hallucinations/Omissions: Versa incorrectly extracts lab data from notes (leading to either a false negative or a false positive). We consider this risk as a special case of both risks defined above as retraining Versa if this occurs would not be possible. While the risk of misreading old results as current is low since we will only add notes from the relevant time period, past labs may still be discussed in those notes. If this risk is observed, we will try to mitigate it through prompting techniques such as metaprompting and chain of thought, or by using a smaller, local model we are capable of fine-tuning, such as ClinicalBERT.

We will further mitigate these risks by tracking real-time use and outcomes; for example, how many flagged alerts led to ordered labs, indicating a lab was indeed past due. We also place a human in the loop, as MAs will interact with the dashboard and report to clinicians and nurses.

5. How Will We Measure Success?

Our plan is to pilot the AI tool in the UCSF Rheumatology clinic during the first year. We aim for a model that is fully implemented and collecting use data by the end of the first year. If effective, we will seek to extend to other UCSF clinics using medications that require toxicity monitoring, such as gastroenterology, dermatology and nephrology. We propose:

During model design and proof of concept (Months 1-3): We will manually chart review a statistically representative sample of our dataset of potential gaps identified by the AI tool with 2-3 UCSF rheumatologists, indicating how many identified gaps were true positives, using classic machine learning metrics (F1, precision, recall, AUROC), as well as correctness of retrieved data. This will also serve as an evaluation set for any future changes to the model. We will continuously add data to this evaluation set throughout pilot testing.

During implementation and pilot testing (Months 4-12):

• Changes in lab completion rates: Proportion of patients on target medications who complete missed labs detected by the AI tool within guideline-specific intervals.
• Number of missed tests identified by Versa missing in structured data.
• Performance of different prompts and models, in clinical media and in notes, using classic ML metrics, including correctness of retrieved lab values from notes and PDFs.
• Time-to-order: Time, in days, from gap detection to lab order and to lab completion.
• Provider feedback: We will interview 2 clinical staff each week for the first week of implementation, followed by every other week until conclusion of the pilot or complete removal of false positives, false negatives, and hallucinations.
• Provider satisfaction: At the conclusion of the pilot test, we will interview 5-6 UCSF rheumatologists and nurses (or until thematic saturation) on their perspective of the AI tool using semi-structured interviews following the technology acceptance model.

If we achieve excellent accuracy (>97% correct assessments) we will explore the possibility of further automating the process by generating pended lab orders for clinicians to sign.

6. Qualifications and Commitment

Project co-leads: Augusto Garcia-Agundez, PhD. Postdoctoral researcher at the Division of Rheumatology. Expert in AI methods and NLP. Dr. Garcia-Agundez will commit 10% effort to implement and validate AI methods and conduct continuous evaluation. Jinoos Yazdany, MD MPH. Chief of the Division of Rheumatology at ZSFG and the Alice Betts Endowed Professor of Medicine at UCSF. Dr. Yazdany has ample experience in quality improvement projects and prior experience designing EHR add-on tools such as a dashboard for Rheumatoid Arthritis. Dr. Yazdany will commit in-kind effort for the year to collaborate with UCSF Health AI and APeX teams. Co-Is: Gabriela Schmajuk, MD, MSc. Chief of Rheumatology at SFVAHC and Professor of Medicine in the Division of Rheumatology at UCSF. Andrew Gross, MD. Dr. Gross is Medical Director of Rheumatology at UCSF. Diana Ung, PharmD, APh, CSP. Dr. Ung is the UCSF Specialty Pharmacist for Rheumatology. Nathan Karp, MD. Director of QI for Rheumatology at UCSF.

7. Summary of Open Improvement Edits

Added clarification about existing preliminary work including branching algorithms for required labs and timeframes, and knowledge of where to pull the required data from Clarity.

The UCSF Health Problem: Endometriosis is a chronic, debilitating condition affecting approximately 10% of reproductive-age women worldwide. It is characterized by the growth of endometrial-like tissue outside the uterus, leading to severe pelvic pain, dysmenorrhea, dyspareunia, bowel and bladder dysfunction, and infertility. Despite its prevalence, endometriosis remains significantly underdiagnosed, resulting in profound patient suffering and substantial economic burden.

The diagnostic journey for endometriosis is protracted, with patients often experiencing a delay of 8-10 years from symptom onset to diagnosis. This delay is compounded by the fact that patients consult an average of 10 physicians before receiving a correct diagnosis. This protracted process leads to years of untreated pain and suffering, impacting patients' quality of life, mental health, and productivity.

The economic impact of endometriosis is substantial. Studies have estimated the annual cost of endometriosis-related healthcare and lost productivity to be in the range of $10,000 to $20,000 per patient. This includes direct medical costs (e.g., physician visits, imaging, surgery, medications) and indirect costs (e.g., lost wages, reduced work productivity, absenteeism). A study published in the Journal of Human Reproductive Update estimated the annual cost of endometriosis in the US alone to be over $69 billion.

Given the significant burden of endometriosis, there is an urgent need for improved diagnostic and management strategies. The current reliance on subjective symptom assessment and non-standardized history taking contributes to diagnostic delays and inconsistencies in care. An AI-driven tool that can standardize patient history collection, assess symptom severity, and predict treatment response has the potential to: reduce diagnostic delays and improve patient access to appropriate care, enhance clinician understanding of endometriosis and improve patient-clinician communication, optimize treatment planning and improve patient outcomes, and reduce the economic burden of endometriosis by minimizing unnecessary healthcare utilization and lost productivity.

How Might AI Help? AI can help by developing a dynamic, interactive questionnaire integrated into the patient's history within APeX EHR. This questionnaire would cover key symptom domains (pelvic pain, dysmenorrhea, dyspareunia, bowel symptoms, etc.), patient medical history, family history, and lifestyle factors. Machine learning algorithms would analyze patient responses to generate: 1. symptom severity score, quantifying the patient's endometriosis burden, 2. risk stratification for specific endometriosis subtypes (e.g., deep infiltrating endometriosis, ovarian endometrioma). 3. personalized recommendations for diagnostic workup, and predicted likelihood of symptom improvement with various treatment options.

How Would an End-User Find and Use It? Within the patient's history in APeX EHR, a "Endometriosis Symptom and Risk Assessment" button would be available. Clicking this button would initiate the interactive questionnaire. Patients could complete the questionnaire in the clinic or remotely via the patient portal. Once completed, the AI would generate a report summarizing the symptom severity score, risk stratification, and personalized recommendations. The report would be displayed within the patient's chart, with visual aids to enhance understanding. Clinicians could use the report to: guide discussions with patients about diagnostic and treatment options, tailor treatment plans based on predicted likelihood of symptom improvement, pend orders for recommended labs and imaging directly from the report interface.

Embed a picture of what the AI tool might look like:

"Endometriosis Symptom and Risk Assessment"

Patient: [Patient Name]

Date: [Date]

Symptom Severity Score: 18 (Moderate-Severe)

Risk Stratification:

- Deep Infiltrating Endometriosis: 60% probability

- Ovarian Endometrioma: 30% probability

Recommended Diagnostic Workup:

- Pelvic MRI with endometriosis protocol

- Pelvic ultrasound

Predicted Treatment Response:

- Hormonal Therapy (GnRH agonist): 70% likelihood of symptom improvement

- Laparoscopic Surgery: 85% likelihood of symptom improvement

[Symptom Severity Graph: Showing patient's pain scores over time]

[Risk Probability Chart: Visualizing risk of different endometriosis subtypes]

[Buttons: “Order MRI,” “Order Labs,” “Discuss Treatment Options”]

What are the Risks of AI Errors? 1. False Negatives: The AI might underestimate symptom severity or miss subtle indicators of endometriosis, leading to delayed diagnosis.False Positives: The AI might overemphasize certain symptoms, leading to unnecessary investigations or treatments, 2. Algorithmic Bias: The AI might exhibit biases based on the training data, potentially impacting care for minority populations.3. Data Misinterpretation: Clinicians might misinterpret AI-generated risk scores or recommendations.

Mitigation strategies include rigorous validation of the AI algorithm on diverse patient populations, clear communication of the AI's limitations and the importance of clinical judgment, continuous monitoring of AI performance and user feedback.

How Will We Measure Success?

This project's success will be evaluated based on the following framework, addressing real-world uptake, meaningful impact, safety, and equity/fairness:

1. Real-World Uptake (Process Metrics): Measurements using data already collected in APeX: include frequency of tool utilization by clinicians (number of assessments completed), time taken to complete the assessment questionnaire, integration of AI-generated recommendations into patient care plans (e.g., orders for recommended labs/imaging), and changes in referral patterns to the Comprehensive Endometriosis Center. Other measurements ideally needed: cinician satisfaction surveys regarding tool usability and integration into workflow, patient feedback on the clarity and helpfulness of the AI-generated report.

2. Meaningful Impact (Health Outcome Metrics): Measurements using data already collected in APeX, such as reduction in the time from symptom onset to diagnosis, changes in the utilization of diagnostic procedures (e.g., number of MRIs, laparoscopies), changes in the utilization of treatment modalities (hormonal therapy, surgery), patient-reported outcome measures (PROMs) for pain, quality of life, and symptom severity, reduction in complications related to endometriosis. Other measurements ideally needed include longitudinal data on symptom control and disease progression, cost-effectiveness analysis of the AI-driven approach, and patient reported measures of improvement in specific symptoms.

3. Safety and Equity/Fairness: Measurements using data already collected in APeX: analysis of potential disparities in tool utilization and outcomes across different patient demographics (e.g., race, ethnicity, socioeconomic status), tracking of adverse events related to diagnostic or treatment decisions influenced by the AI tool. Other measurements ideally needed: validation of the AI algorithm's performance on diverse patient populations to ensure equitable outcomes patient feedback on perceived fairness and trust in the AI-driven assessment.

Evidence for Continued Support/Abandonment: Continued Support: Significant reduction in time to diagnosis, demonstrable improvement in patient-reported outcomes, and high clinician/patient satisfaction would warrant continued support. Abandonment: Evidence of significant algorithmic bias, lack of clinical uptake, or adverse patient outcomes would necessitate re-evaluation or abandonment.

Describe your qualifications and commitment: This project addresses a high-priority area within UCSF Health, specifically the significant delays and disparities in endometriosis diagnosis and management. The AI-driven tool has the potential for large-scale impact by improving patient outcomes and streamlining clinical workflows.

Jeannette Lager, As the Section Chief for Minimally Invasive Gynecologic Surgery (MIGS) at UCSF, and formerly the Medical Director and Interim Chief of the Urogynecology and MIGS division, I have clinical and leadership expertise directly relevant to endometriosis care. My role as Associate Director of the Comprehensive Endometriosis Center provides me with deep insight into the patient population and the challenges they face. I have a strong understanding of the clinical workflows and can design the AI tool to integrate seamlessly into existing practice.

My co-investigator, Zaineh Khalil, is a highly experienced Nurse Practitioner with a specialized focus on endometriosis care. Her extensive clinical experience, coupled with her deep understanding of patient needs, makes her an invaluable asset to this project. She has been actively involved in quality improvement initiatives within the MIGS and Urogynecology clinic. Her expertise in navigating the complexities of endometriosis management, combined with her understanding of clinical workflow, will be instrumental in ensuring the tool's practical application and successful integration into the clinical setting.

Furthermore, I have a strong track record of collaboration with UCSF administration and faculty, ensuring engagement and support from critical operational and clinical champions. Together with my co-investigator, we are positioned to effectively lead this project and ensure its successful implementation.

The UCSF Problem:

Lung nodules are a common finding on CT and are often benign but rarely may represent malignancy. To monitor for potential malignancy, follow-up imaging is often recommended. If there is concern for malignancy, adequate follow-up allows earlier diagnosis and treatment, while lack of follow-up may lead to progression and worse prognosis.  

Lung nodules are found in two different ways: lung cancer screening CTs, which are CT scans to screen patients who are at high risk for lung cancer, and those incidentally found on other CT scans. At UCSF, we perform around 400 screening CTs a year, a number that is dwarfed by the number of incidentally found nodules. Approximately 65,000 CT chests for other indications were performed in 2024, and population data has noted that around 31% of CT chest scans contain a nodule. At UCSF around 4,000 CTs in 2024 have the word “nodule” or “follow-up” in their impression¹. While not all of these may represent lung nodules requiring follow-up, the number is likely in the thousands.

Here a UCSF, we have a lung nodule nurse coordinator to follow-up nodules found on lung cancer screening CTs. These are tracked on an Excel spreadsheet. Unlike many peer institutions such as the VA and SFGH, we have no system-wide tracking program for incidentally found nodules. Concerning nodules must be tracked by the ordering physician or a patient’s primary care doctor, which may be challenging without a centralized system for tracking these nodules and risks a patient not obtaining adequate follow-up.

How might AI help?

Manually reviewing all 65,000 CT scans per year to find concerning nodules would be time-consuming and cost prohibitive. However, large language models could efficiently and cost-effectively parse through reports and extract the key information needed to determine the optimal follow-up time. This information can then be used to populate a dashboard, enabling coordinators to track suspicious nodules, identify if a patient is overdue to follow-up, and easily reach out to patients.

How would an end-user find it and use it?

Our AI tool would be limited to users part of the pulmonary nodule program including lung nodule coordinators and pulmonologists. We will directly coordinate with the relevant individuals so that they are aware of and understand how to use the AI-enabled dashboard.

Picture of the AI tool:

Using Versa API, we will extract the information needed for clinical decision-making about lung nodules from radiology reports. This includes the number of nodules and recommended follow-up time if suggested by the radiologist. For nodules 6mm or greater, which are considered higher risk, we will extract the size of the nodule, location, and characteristics such as groundglass, solid, or subsolid as these would change the recommended follow-up time as per the guidelines. This data will be extracted as structured data using OpenAI’s structured outputs and then be used to build a dashboard. The dashboard will contain key patient information, including age, MRN, date of CT, characteristics noted above, if a repeat CT is scheduled and when, and if a referral to a relevant subspecialty is ordered as shown in Figure 1. This will allow for easy filtering of which patients have been connected to care and which patients may benefit from outreach.

Figure 1: Mock-up of dashboard 

What are the risks of AI errors?

There are two primary risks to introducing AI for lung nodule tracking:

1. A significant nodule requiring follow-up is not reported: The nodule would then not be included in the monitoring dashboard. However, CT scan results would be sent to the ordering user for follow-up. At present, incidental lung nodules are not tracked and so this workflow would be the same as the current state.
2. Inaccurate description of the nodule or its characteristics: It is possible that the model may describe a nodule when one does not exist or incorrectly state the characteristics of the nodule. The dashboard will include a copy of the original report to verify that the nodule exists and the characteristics of the lung nodule.

 To mitigate against these risks, we will perform robust testing prior to launch on retrospective data to assess accuracy and identify any characteristics that lead to a higher risk of hallucination.

Our preliminary testing on 200 CT chest reports from 2024 containing the word nodule notes a 96.4% accuracy and 1.5% hallucination rate. We achieved a 95% sensitivity and 97.1% specificity in identifying nodules 6mm or greater, a key cutoff point in the lung nodule guidelines and when estimating the risk of malignancy. Our precision was 93.5% and F1 score was 94.3%. Continuous performance monitoring will be performed to ensure persistent high accuracy and hallucination rate. A lung nodule coordinator will also have access to the full CT scan report and will serve as the human-in-the-loop to verify all characteristics.

How will we measure success?

We will measure success by:

-       Sensitivity and specificity: We will assess the sensitivity and specificity of the model in detecting nodules requiring follow-up.

-       Percentage of patients who obtain recommended follow-up images: We will assess what percentage of patients with lung nodules obtain the recommended follow-up imaging within 3 months of the recommended time frame.

-       Frequency of outreach: We will track how often the lung nodule monitoring team reaches out to patients with lung nodules.

-       Qualitative feedback: We will obtain feedback and measuretool satisfaction, usability, helpfulness, and perception of accuracy from those involved in the lung nodule monitoring program

-       Time to referral: We will measure time to referralfrom identification of the nodule to the next recommended step of care, including interventional pulmonary, oncology, thoracic surgery, and interventional radiology.

-       Equity and bias analysis

Describe your qualifications and characteristics:

Cat Blebea, MD: Dr. Blebea is a current pulmonary and critical care fellow and clinical informatics fellow. She frequently manages patients with pulmonary nodules in clinic and has first-hand experience with pulmonary nodule monitoring programs at the VA and SFGH. She also as experience using large language models to improve patient care as a member of the prompt engineering team for the Intelligent InBasket project, which uses large language models to create draft responses to patient messages.

Leo Liu, MD: Dr. Liu is an informaticist and hospitalist at UCSF. He serves as the physician lead for inpatient informatics at St. Mary’s and St. Francis, associate program director for the clinical informatics fellowship, and director of the GME Clinical Informatics, Data Science and Artificial Intelligence Pathway.

References:

1.         Gould MK, Tang T, Liu ILA, et al. Recent Trends in the Identification of Incidental Pulmonary Nodules. Am J Respir Crit Care Med. 2015;192(10):1208-1214. doi:10.1164/rccm.201505-0990OC

Summary of Open Improvement Edits: 

Updated proposal with recent preliminary data findings. 

1.  The UCSF Health problem.   

Diagnostic errors are common and harmful. Previous work from our group suggests a missed or delayed diagnosis takes place in 25% of patients who die or who are transferred to the ICU; a diagnostic error is the direct cause of death in as many as 8% of deaths.

Our research team, as part of a multicenter study called Achieving Diagnostic Excellence through Prevention and Teamwork (ADEPT) led by UCSF has been prospectively gathering information about delayed or missed diagnoses at UCSF Health since 2023, focusing this time on RRT calls, ICU transfers, and deaths on the Medicine service.  ADEPT randomly samples 10 cases per month, using a two-physician adjudication approach to develop gold-standard case reviews which yield not only an assessment of whether an error took place but also the harms related to the error, and any underlying diagnostic process faults.

Our current data suggest a significant opportunity for improvement, with 12% of patients who died, had an ICU transfer, or RRT call having a diagnostic error.  Harms were substantial – with 17% of errors thought to be a cause of death, and 40% producing temporary or permanent harm (such as need for additional monitoring or testing, longer length of stay, or additional therapies).   The most common diagnostic process fault was problems with assessment (e.g. anchoring on a diagnosis, or failing to recognize a stronger alternative), testing problems (not choosing the right test in a timely fashion).  If we were to extrapolate the results from ADEPT to all 17000 admissions to UCSF Health, these numbers would represent more than 500 errors per year, of which 83 were a likely cause of death.  Even if not causing death, 200 patients would have suffered longer hospital stays and additional and potentially unnecessary treatments.

2. How might AI help? 

The field of patient safety in general, but diagnostic errors in particular, suffers from an inability of health systems to detect errors and opportunities for improvement at scale.  Uniquely, diagnostic errors are extraordinarily hard to detect using administrative data, and determining underlying causes with any accuracy requires chart reviews such as our team carries out.  AI tools, and LLM’s in particular, are a potentially groundbreaking approach to solving both problems.

Our current chart-based approach, while appropriate for research purposes as part of our ADEPT AHRQ-funded work, is time-consuming – taking up to 45” per case to finalize.  A lighter-weight approach could be folded into existing case review programs (such as M&Ms) but would still need substantial support in determining underlying causes and developing an actionable set of priorities for leaders in Safety, or providers seeking to improve performance. 

Preliminary data: We have recently begun work to develop llama-3 and GPT-4o LLMs with prompts derived by ADEPT diagnostic process review team  (Figure).  Prototype ADEPT Diagnostic Process Reviews (ADPR) utilize llama-3 and GPT-4o models (running in secure AWS cloud environment) against a Clarity-derived data model composed of all notes, results, orders, and vital signs (including intake/output) for the hospital stay, concatenated.  Preliminary LLM-ADPRs are producing results that replicate our case reviews but are fairly unsophisticated; we are undertaking work now to improve clinical utility, and to determine correlation between our reviewers’ assessment and those of the LLM-ADPR.

Project plan:  We propose to leverage the expertise of our case review teams, a large sample of more than 300 ‘gold standard’ cases, and pilot work we have currently underway using llama-3.1 and GPT-4o models leveraging EHR data gathered as part of ADEPT, to create an AI-powered diagnostic excellence learning cycle targeting Medicine patients.

By using ADEPTs existing case review process as a starting place to engineer prompts that replicate clinical adjudication and also monitor LLM-ADPR agreement with expert and users’ assessments, we will be able to generate diagnostic process-focused case summaries that replicate the evidence-based framework used in our research studies but at much lower time requirements. These summaries can then be used for at least three purposes:

1) As part of an enhanced UCSF mortality review process, with the LLM summary added to existing reviews as a way to identify opportunities for physician and system performance improvement,

2) As part of an automated diagnosis cross-check/diagnostic time out provided electronically to clinicians after an RRT takes place.

3) As part of an automated self-assessment and feedback system employed after an ICU transfer, or patient death has taken place. This summary may be presented in individual cases, or as a summary of all events which took place during a clinical block.

As a first step, we will map our current data model into HIPAC data sources, and then link HIPAC EHR data to data from our chart reviews to create our validation dataset (we anticipate 375-400 cases being available.  Using these data we will iteratively test prompts against selected encounters, while also identifying methods (e.g. RAG) for dealing with longer hospital stays. Once we achieve a high level of clinical reliability and accuracy in the derivation set, we will proceed to deploying enhanced Mortality Review, diagnostic cross-check, and self-assessment pilots focused on Medicine patients and leveraging the existing partnerships between our ADEPT team and UCSF Health leadership. Emblematic of this partnership: our ADEPT review group is now a subgroup of the UCSF Patient Safety Committee and ADEPT cases with harmful errors are submitted as Incident Reports.

3. How would an end-user find and use it?  

We do not plan to embed these case summaries in the medical record but would generate ADEPT-LLM diagnostic process summaries as part of existing case review processes (option 1, above), or deliver them linked to a REDcap survey within 12 hours of an RRT call (Option 2, above), or via REDCap survey 1 week of ICU transfer or death (Option 3). For each use case, the LLM-ADPR will provide a framework for reconsidering the case and diagnostic process, along with survey questions which will permit the end-user to agree or disagree with the summary and offer clarifications.

4.  Embed a picture of what the AI tool might look like. 

A preliminary version of the LLM-ADPR summary is provided; the LLM-ADPR would be accompanied by a REDCAP survey (or embedded in the survey, itself).  A key step in our pilot/validation step will be to increase usability ofour LLM-ADPR output in addition to clinical accuracy.

5. What are the risks of AI errors?    

Given that our AI output will not be used to actively direct clinical care, risks of hallucinations (e.g. false positive results), or unclear/imprecise results on patient care are nearly zero.  However, it is possible that these same issues could produce higher work burden on patient safety reviewers or end recipients. 

We do not envision this case review summary being represented in Apex but presented to providers and Patient Safety review staff separately from clinical workflows. This design increases the safety of our program, increases feasibility (since compute power for entire encounters is limited, particularly for real-team computations), and maps towards a future state where diagnostic cross-check and case feedback tools can be inserted into workflows or clinical scenarios where context length, clinical need, and patient factors increase overall effectiveness.

6. How will we measure success?   

While the ultimate goal of this program is to reduce diagnostic errors, harms of diagnostic errors, and in turn mortality, ICU transfers, and RRT calls, it is unlikely we will show an effect on these outcomes in a year.

For this study period, we will measure agreement between LLM-ADPR documents and our recipient (e.g. patient safety staff, clinicians whose patients have a trigger event) in terms of their agreement with the identified opportunities for diagnostic improvement, as well as usability of the LLM-ADPR.  In the 10-12 cases/month reviewed by ADEPT case reviewers (ADEPT will run through September 2026), we will be able to calculate agreement statistics compared to gold-standard adjudications.  Finally, we will estimate the work-hours saved for patient safety staff and clinical reviewers, time which can eventually be better used to address care gaps rather than in gathering chart review data.

7. Describe your qualifications and commitment:

This program will be led by Andrew Auerbach MD MPH, PI of ADEPT and a leader of several informatics and implementation science-focused programs at UCSF, as well as our existing ADEPT Project Coordinator Tiffany Lee BA.  Our team includes our current ADEPT review team (Drs. Molly Kantor, Peter Barish, Armond Esmaili), as well as informatics and LLM experts (Drs. Charumathi Subramanian and Madhumita Sushil);  Dr. Sushil led development of the LLM-ADPR shown in Figure 2. Finally, our program has strong endorsement and support of Chief Quality Officer Dr. Amy Lu.

The UCSF Health problem: Cancer survivors at UCSF Health have unmet needs significantly impacting their health-related quality of life (HRQOL). These include managing ongoing side effects of treatment, surveillance for cancer recurrence, informational needs, health promotion and coordination of care. A Survivorship Care Plan (SCPs) is a document that is completed at the completion of curative cancer treatment when the patient is transitioning to survivorship care. SCPs are a patient activation tool that provides patients with important information about the details of their cancer care, follow-up plan, delayed and long-term side effects, and health maintenance. SCPs are considered an important component of comprehensive survivorship care by NCI(1) and Commission on Cancer (COC) that accredit cancer programs. Furthermore, SCPs can also serve as communication tool between primary care and oncology(2). However, the completion of SCP by healthcare providers has been low despite the fact that cancer survivors and primary care physicians (PCPs) consider SCP as an important tool in addressing informational needs of cancer survivors. At UCSF Health most patients do not get SCPs. The major barriers to SCP implementation are: 1. Creation of SCPs is time-consuming for the healthcare provider, 2. These complex documents require manual entry of many details leading to possible inaccuracies, and 3. SCPs are static documents and cannot be easily tailored to cancer types and include updated guidelines with advances in cancer care(3). Current literature suggests that it takes approximately 40 minutes to create a survivorship care plan, and that leaves very little time for discussion of the actual plan in a clinic encounter(4). The discussion between patient and provider of the SCP is the most important part of SCP process which is often unfortunately shortchanged in SCP delivery(5).

How might AI help?  AI/LLM are increasingly being used in many aspects of health care including medical documentation, radiology and pathology diagnostic reporting, patient communications, and decision supports. We expect that the time-consuming process of SCP/ treatment history generation could be taken over by AI, which will allow clinicians to spend more time with patients on discussion of SCP. We can optimize EPIC tools to provide automation of patient information such as demographics, health care team members, diagnosis and staging, and treatment received. We would utilize AI to provide multiple patient-friendly personalized components of the SCP. These components would include: 1. Long-term side effects of specific treatments received and of the cancer diagnosis itself (eg: cardiotoxicity, neuropathy, lymphedema, fear of recurrence), 2. UCSF resources relevant to the patients' diagnosis and demographics, including website and phone numbers, 3. Current NCCN Surveillance guidelines, based on patients' diagnosis, 4. Risk reduction strategies and resources based on the patients' diagnoses, 5. USPTF Age-appropriate healthcare maintenance screening recommendations based on the patients' demographics such as age and gender. We expect that this will include information from reputable organizations such as American Cancer Society, NIH, NCCN, USPTF and local UCSF resources. We expect that AI may be able to create these plans in five to ten minutes instead of 40 minutes allowing encounter to focus on discussion between patient and provider.
How would an end-user find and use it?
The SCPs  will be created by Survivorship/Oncology Healthcare providers who will be trained in the AI enabled process. The SCP will be delivered in the clinic and will be available to patients via MyChart. We anticipate SCP to be accessible to all providers and in the Survivorship section on the oncology snapshot page and attached to the problem in the Problem List. AI will allow these plans to be dynamic and easily updated with the most current resources and guidelines. AI support will be most impactful at the time of the SCP creation, which will be delivered at transition to survivorship care after completion of treatment for patients treated with curative intent. We expect two sets of end users. The first set will be patients who will be sent the SCP via My Chart and we aim to empower patients to manage their surveillance schedule, health care maintenance, and take steps to optimize their wellbeing. The other end user will be the PCPs who will be sent the SCP via MyChart or fax and will utilize the information to provide tailored appropriate primary care to cancer survivors. Finally, we hope that AI enable process will allows us to easily translate these into multiple languages allowing more patients to benefit from this process.

Embed a picture of what the AI tool might look like.   

We anticipate two distinct steps in the creation and implementation of the tool. Step 1 will include knowledge base creation step in which we will create a knowledge base to add to existing data in EPIC/APeX which will include key guidelines for cancer survivorship, relevant medical literature, and chemotherapy database including expected side effects and management of side effects(6). Based on the current literature we will divide this knowledge into the domain of survivorship care including 1. Surveillance, 2. Management of side effects of chemotherapy, 3. Health maintenance, 4. Management of comorbidities, and 5. Coordination of care. Step 2. will be creation and implementation of tool, in this step once clinician identifies a patient for SCP creation, the APeX tool will prepopulate the SCP template with details of treatments, and side effects from problem list, and the AI-enabled SCP will be created using the APeX data and knowledge base. Once the SCP is generated, the clinician will verify its accuracy and share it with the patient. We also expect a task-creation step including surveillance imaging, screening for cancer, and nudges for referrals for services. We will continue to iterate the SCP creation and delivery process to make it patient friendly and helpful to PCPs. We will actively seek feedback from survivorship expert group at UCSF and a patient advisory board to improve the SCP. We expect the SCP to be available in the Oncology Snapshot or problem list in APeX (see below). For patients, it will be available through MyChart.

 Once the SCP generation process is standardized, we will generate and deliver the first 20 SCPs and will evaluate the process with our survivorship expert group and patient advisory board. We will iteratively improve the process to optimize he SCP generation and delivery process.

What are the risks of AI errors?  The biggest risk would be hallucinations, as AI may generate recommendations that are not evidence-based. We will train clinicians who will participate in the pilot to double-check recommendations and links to ensure that recommendations are within guidelines. Site-specific survivorship providers will vet all initial AI generated SCPs to ensure that all SCPs adhere to the guidelines for surveillance and follow up. Finally, we will do a set of 20 SCPs which will be assessed for accuracy and consistency by investigators.   

How will we measure success?  We will use specific metrics to evaluate success. 1. Evaluation of AI-SCP for accuracy and consistency by the investigators and end users, 2. The number of SCP created and delivered to patients, 3. The time needed to create SCP’s. Delivery of SCPs will help us meet the NCI and  COC requirements for Cancer Survivorship Care. Finally, we will evaluate the SCP implementation success using the four item measures of Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM)(7). This will be done as a brief survey for providers creating the SCPs.  

6.A. Measurements below are already being collected in APeX  

1. Number of patients eligible for SCPs: those treated with curative intent 

2. SCP delivery: the numerator is patient receiving SCPs, the denominator is eligible patients. We will evaluate current overall numbers, and site-specific data. What disease sites are not using the SCP currently.  

3. Capturing treatment details including prior chemotherapy, surgery, radiation oncology.  

4. Capturing prevalence of persistent side effects.   

6.B. A list of other measurements you might ideally have to evaluate success of the AI  To evaluate the success, we will need to include measures such as cancer surveillance, HRQOL measures, needs assessment, comorbidity care measures such as Hemoglobin A1C, lipid profile, screening for other cancers in the last one year for patient who have received SCPs.   

4. Describe your qualifications and commitment: Our applicant team includes Dr Niharika Dixit and NP Angela Laffan and cancer survivorship provider group. Dr Dixit is a medical oncologist and Professor of Medicine focused on breast cancer and survivorship care. She provides clinical care at ZSFG and is the Physician-lead of the UCSF Survivorship and Wellness Institute. Dr. Dixit focuses on improving care of cancer survivors across UCSF and Affiliate sites. She has conducted several research projects related to cancer survivorship and cancer survivorship care planning and is committed to optimal delivery of cancer survivorship care using innovative approaches in diverse clinical settings. Dr Dixit additionally serves as co-chair of a Cancer Survivorship Task Force for ASCO and is the co-lead for the UCSF Survivorship and Symptoms Science Research Hub promoting research in cancer survivorship and symptom science. NP Laffan is an Oncology Nurse Practitioner in the GI Medical Oncology and GI Survivorship program at UCSF. NP Laffan is the co-creator of the GI Oncology Survivorship Program and has an active clinical practice providing survivorship care to patients who have completed treatment for GI related cancers.  NP Laffan is also the Clinical Lead in the UCSF Survivorship and Wellness Institute focusing on program development, survivorship research initiatives and clinician education. We are also supported by other survivorship care clinicians who have site specific expertise such as Lung, breast and colon cancer etc. This group meets who meet once a month to discuss survivorship care across UCSF. We will present our planned intervention in these meeting to solicit ongoing feedback, As part of this endeavor we will work with a Patient advisory board to iteratively refine SCPs creation and delivery. Together, we believe that we are a strong team with extensive knowledge of survivorship care including barriers and exciting areas of opportunity. We are energized by this exciting opportunity to work with the health AI team and APeX enabled research team to address a health care issue that has been limited predominantly by the time intensity and complexity of the task. We are confident that AI can assist in rapidly creating SCPs that are patient friendly, easy to access, tailored and actionable which will lead to improved survivorship patient care.

References:  

1.    Mollica MA, McWhirter G, Tonorezos E, Fenderson J, Freyer DR, Jefford M, et al. Developing national cancer survivorship standards to inform quality of care in the United States using a consensus approach. J Cancer Surviv. 2024 Aug;18(4):1190–9.

2.    Hayes BD, Young HG, Atrchian S, Bennett EV, Haynes EMK, Loader A, et al. Optimizing the integration of family physicians into cancer survivorship care in the BC Interior: a mixed methods study of physicians’ opinions and experiences. J Cancer Surviv. 2025 Feb 4;

3.    Birken SA, Deal AM, Mayer DK, Weiner BJ. Determinants of survivorship care plan use in US cancer programs. J Cancer Educ. 2014 Dec;29(4):720–7.

4.    Birken SA, Mayer DK, Weiner BJ. Survivorship care plans: prevalence and barriers to use. J Cancer Educ. 2013 Jun;28(2):290–6.

5.    Dixit N, Sarkar U, Trejo E, Couey P, Rivadeneira NA, Ciccarelli B, et al. Catalyzing Navigation for Breast Cancer Survivorship (CaNBCS) in Safety-Net Settings: A Mixed Methods Study. Cancer Control. 2021;28:10732748211038734.

6.    Pradeepkumar J, Pankaj Kumar S, Reamer CB, Dreyer M, Patel J, Liebovitz D, et al. Survivorship Navigator: Personalized Survivorship Care Plan Generation using Large Language Models. medRxiv. 2025 Mar 28;

7.    Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017 Aug 29;12(1):108.

The UCSF Health Problem:

Sedation planning prior to endoscopic procedures is an important quality metric and essential step in the procedural workflow. Triage of which patients require higher levels of anesthesia support is critical to maximize patient safety and allocate limited anesthesia resources. At most institutions, patients are assessed for sedation risk from pre-existing medical conditions based on manual chart review by clinical staff and endoscopists: this is a time consuming and labor-intensive task.  

At UCSF Health, patients are triaged for endoscopy procedures across five locations, based on their anesthesia risk and required sedation type. As our health system expands across hospital systems and ambulatory care centers throughout San Francisco and the greater Bay Area, the decision tree for appropriate triage becomes increasingly complex. Currently, the gastroenterology office reviews up to 150 direct endoscopy referrals per week. This workflow of manual review by staff and faculty diverts valuable time away from direct patient care and contributes to administrative burdens, which in turn leads to physician burnout. 

Although this proposal is initially tailored to gastroenterology-specific procedures, the administrative challenges of peri-operative sedation triage are widespread across the health system in many divisions, highlighting the larger potential uses for an AI-based sedation triage assistant. 

How might AI help?

We propose the development of an LLM-based assistant (UCSF Versa) customized via the Retrieval-Augmented Generation (RAG) methodology. This approach adds the UCSF anesthesia endoscopy guidelines and American Society of Gastrointestinal Endoscopy (ASGE) clinical guidelines into the assistant’s search database, thereby reducing hallucinations by controlling the data sources used for response generation. This customization named, “GIVersa-Endoscopy” would serve as the chat interface intended to triage sedation levels (moderate sedation versus anesthesia) and recommend the appropriate procedure location stratified by anesthesia risk level (Parnassus operating room, Parnassus endoscopy unit, Mission Bay endoscopy unit, Mount Zion endoscopy unit, or ambulatory care centers) based on clinical patient data extracted from the electronic medical record. This design has been successfully tested by our team in a pilot retrospective cohort study using a custom Epic Smartphrase to extract relevant clinical data for the assistant. 

How would an end-user find and use it?

The GI-Versa Endoscopy LLM assistant will be integrated into the existing Epic referrals and pre-procedural planning dashboards. When processing a direct procedure referral, the end-users, the administration staff and faculty member responsible for reviewing and scheduling new referrals, will receive an alert with an option to generate the AI-augmented triage recommendation. The interface will allow the user to “Approve” or “Decline” the AI’s recommendation. The final decision on sedation level and where the patient should be scheduled for their procedure along with the underlying reasoning on why the assistant reached that decision will be recorded in the patient’s chart. This integration ensures that the tool is actionable, easily discoverable, and seamlessly fits into the existing workflow.  

Example AI tool output

See attached figure. 

What are the risks of AI errors?

The risks for an AI-augmented preprocedural sedation triage assistant includes: 

1) Patient safety: A false negative would occur when a patient is recommended a lower level of sedation support by the assistant when the patient should have been triaged to a higher level of sedation support. 

 2) Overdependence on the AI: Excess reliance on the AI system may compromise decision making in complex clinical cases. 

To mitigate these risks, we will conduct rigorous validation tests and continuously monitor performance metrics such as sensitivity, specificity, and error rates. The system is designed to require human verification of the AI’s recommendations prior to scheduling the patient.  All AI assistant outputs will be logged in the patient’s chart for audit and review. Licensed professionals will maintain primary accountability for patient care decisions. 

How will we measure success? 

We plan to evaluate the AI solution through a prospective cohort study of patients referred for direct endoscopy at UCSF Health. Success metrics will include: 

Measurements Using Existing APeX Data:

1)    The volume of referrals processed by the AI tool.

2)    The percentage of referrals where AI recommendations are accepted after clinician review.

3)    Reduction in manual review time as recorded within current workflow data. Time reduction in manual processing of each direct procedure referral.

4)    Correlation between AI-augmented sedation risk stratification and patient outcomes.  This will be measured via post-procedural audits which are performed to ensure successful and safe completion of patient procedures. This workflow is already in place for endoscopic procedures at our health system. 

Additional Measurements:

1)    Review of AI recommendation logs versus human overrides to identify error rates and opportunities for improvement in assistant performance. 

2)    End-user satisfaction surveys to assess overall usability, reduction in administrative load, and time efficiency. 

3)    Impact on patient wait times to schedule procedures (time from referral placement to receipt of procedure appointment confirmation). 

4)    Impact on administrative clinic staff and ease of integration within existing workflows.

Describe your qualifications and commitment 

Dr. Lakshmi Subbaraj, MD, a current second year gastroenterology fellow, has career aspirations in the integration of AI applications in clinical practice to enhance patient care and improve clinical operations. Her technical background in computer science, completion of an “AI in Healthcare: From Strategies to Implementation” course, and recognition as an ASGE AI scholar underscore her commitment to this goal this early in her career. The blend of her clinical acumen, technical expertise, and communication skills has been pivotal in spearheading this project.

Dr. Jin Ge, MD, MBA, an NIH-funded clinical researcher and transplant hepatologist, also serves as Director of Clinical AI for the Division of Gastroenterology and Hepatology. He is the co-lead for this project and has a proven track record in building and implementing AI initiatives. His professional background with experiences in healthcare administration, data science, and artificial intelligence – along with his previous collaborations with the UCSF AER Team and AI Tiger Team, make him exceptionally qualified to lead this project through the design, testing, and implementation stages. 

Section 1. The UCSF Health Problem

While pain after spine surgery is expected to resolve within a few weeks, a substantial number of patients experience post-operative pain for over 6 months¹. At UCSF Health, nearly 15% of patients need opioids three months after spine surgery—an indicator of this chronic post-surgical pain (CPSP) that adds significant burden to both the surgical care team and the broader health system. When poorly managed, CPSP after spine surgeries leads to new or escalated chronic opioid use², higher costs of care³, and worse post-operative outcomes⁴. However, only a small fraction of these patients at UCSF have been evaluated by the pain service for either pre-surgical optimization or post-operative pain management.

To address this problem, the Pain Medicine Department launched a transitional pain service, a specialized program providing multidisciplinary perioperative care with a tailored combination of preoperative optimization, perioperative pain management planning, and ongoing support during the outpatient transition. Several other institutions have successfully implemented the TPS, resulting in significantly improved pain control and opioid use in targeted patient cohorts^5-7. However, given the limited capacity of this engaged service, a better method to select high-risk patients is essential to maximize the efficacy of a TPS⁸.

Current perioperative guidelines provide simple selection criteria, such as the O-NET+ classification, that rely on a few indicators like psychiatric conditions or previous opioid use to guide patient selection⁹. Given the interplay of the various biopsychosocial factors that contribute to the development of CPSP, these criteria fall short in identifying vulnerable patients with modifiable risk factors. An intelligent selection mechanism is needed to help our intended end-users—UCSF neuro-spine and pain providers—easily evaluate and identify high-risk patients for referral to the UCSF TPS that can improve patient outcomes and reduce health system costs.

Section 2. How might AI help?

Given the constrained clinical capacity of a TPS, it is essential to identify patients with the highest likelihood of developing CPSP. Excessive enrollment of patients at low risk of developing CPSP can dilute TPS efficacy, allocating care to patients unlikely to benefit and diverting resources from patients with true CPSP. We believe that an artificial intelligence (AI) tool is the best approach to identify TPS patients as AI can automatically analyze multiple factors simultaneously and provide clinicians with a more comprehensive risk profile. While numerous machine-learning models have been developed to predict post-surgical opioid use and pain trajectories¹⁰, none have been rigorously evaluated for their ability to guide referrals to a TPS.

We previously retrospectively validated a decision-tree-based model using an eXtreme Gradient Boosting (XGBoost) algorithm with Neuro-Spine cases at UCSF from 2015 to 2023 at the pre-operative time of the surgical booking date. This model analyzes data pulled from UCSF Clarity, incorporating important features such as procedural details, prior medications, prior pain-related ICD diagnosis codes, and clinician prescribing patterns. The target population includes patients who were prescribed opioids between 30- and 180-days post-discharge and exhibited severe acute post-surgical pain trajectories, as determined by a simple linear regression classification of their in-patient pain score trajectory. This outcome definition was developed with pain physicians as the ideal TPS population, and we are now validating this rule-based criterion retrospectively against pain providers' clinical assessments of post-surgical pain syndrome. Using this outcome, the model showed superior performance at identifying patients with challenging postoperative pain control and opioid usage at the surgical booking date when compared to the O-NET+ tool from perioperative guidelines⁹ (Figure 1). For comparison, the positive predictive value was 0.97 with our XGBoost model and 0.43 with the O-NET+ tool at the 0.68 threshold selected based on prospective validation results (Figure 5), highlighting our model’s superior specificity in identifying true high-risk patients compared to broader, less discriminating traditional criteria. Using the AI Pilot Award, we aim to silently prospectively validate this model to demonstrate that our model better identifies high-risk Neuro-Spine patients for TPS referral in the clinical setting compared to traditional methods.

After confirming that the accuracy of prospective validation matches the retrospective results, we plan to initiate an intervention using model-generated probabilities to refer Neuro-Spine patients to the previously established UCSF TPS, collaborating with UCSF pain physicians to conduct a pilot randomized controlled trial of an AI-driven TPS. The model is critical to this trial’s success because a prior RCT of the TPS⁷ lacked power for long-term outcomes, largely due to inadequate patient identification using basic screening methods. In the trial, patients receiving the TPS intervention will be compared to standard-of-care controls matched on outcome probabilities predicted by the model. Assuming a conservative estimate that 50% of the patients identified by the model will develop post-surgical pain, a stratified power analysis with an alpha of 0.05, 80% power, and a 10% absolute event reduction indicates that 104 participants per arm are needed. (Figure 2). Ultimately, we hope that this pilot randomized controlled trial of an AI-enabled TPS for UCSF Neuro-Spine patients will generate evidence of the clinical safety, equity, and benefit of our approach.

Section 3. How would an end-user find and use it?

The model will be embedded into the clinical workflow by running the model each night on all surgical cases booked that day for future surgical dates, using real-time data from UCSF's Epic Clarity Instance. When a patient exceeds the risk threshold, an automated notification will be sent to both the neuro-spine surgical team and the TPS team. The exact notification method—whether via email, APeX in-basket message, secure chat, or phone call—will be determined in collaboration with the development team and through planned user testing. Regardless of the modality, the notification will deliver a TPS referral recommendation and support pre-surgical planning for patient optimization. As an interpretability component of model results to promote clinician oversight, the notification will include a report generated by UCSF Versa GPT, a PHI-secure large language model (Figure 3).The report explains the rationale behind the model’s prediction using LLM-generated descriptions of the most important features for that patient. The surgical and TPS team will review this information and act on its recommendation—specifically, by initiating a referral to the TPS and coordinating perioperative pain management strategies. This low-friction, automated approach ensures the AI support is easily discoverable, actionable, and aligns with existing clinical workflows.

Section 4. Embed a picture of what the AI tool might look like.

In APeX, we plan to have a simple interface that includes the predicted risk score along with the key features driving the prediction (Figure 4A).While this approach will support clinical decision-making around the patient’s risk for CPSP, we also ideally envision an interactive, in-depth dashboard that will require more resources in implementation (Figure 4B). This dashboard includes three core components: (1) a visual risk bar indicating the predicted CPSP risk level, (2) a direct-action link enabling clinicians to place an order for TPS referral when appropriate, and (3) a display of the top features influencing prediction for each patient, explained to the clinician by the LLM-generated report, with the option to adjust key variables and re-run the model as needed.

Section 5. What are the risks of AI errors? 

Several types of AI errors could impact the effectiveness of our model and an AI-driven TPS. First, Classification Errors. Excessive false positives may dilute the TPS’s impact by allocating resources to lower-risk patients, and false negatives may inappropriately reassure care teams, leading to undertreatment of pain and missed patient concerns. We plan to monitor month-to-month variations in model performance, identifying features or patient clusters that may contribute to a less-than-optimal performance using our previously published toolkit of statistical methods¹¹. To avoid confounding from the presently active TPS, we will also exclude TPS-referred patients from prospective validation; their low volume makes them unlikely to bias results. Second, Inequities in Model Performance. Disparities in prediction accuracy across race, gender, or opioid history could limit equitable access to care. We will track performance metrics stratified by these groups and aim to keep PPV differences within 10%. If differences exceed this threshold, we will address them using the statistical toolkit stated above. Third, Poor Calibration. A model that poorly balances opioid-naïve and opioid-experienced patients may reduce TPS effectiveness. Using the distribution of prediction scores of the prospective validation cohort, we can simulate month-to-month sampling to extrapolate how fluctuations in performance and cohort make-up will affect our likelihood of showing a statistical benefit. Using the effectiveness of previous TPS trials, this simulation can inform a power analysis for a pilot TPS trial and its predicted efficacy (Figure 5).

Section 6. How will we measure success? 

We designed specific success criteria for the prospective validation of the model and the clinical trial. For model validation, measurements can be derived using data that is already being collected in APeX. The primary endpoint for clinical readiness is achieving a sensitivity of at least 0.4 while maintaining a PPV of at least 0.67. Currently, we far exceed the goal PPV and sensitivity at a threshold of 0.68, which allows us to flag a minimum of 8 patients per month. Secondary endpoints include: (1) chart review confirmation by the pain service of patients flagged as positive, (2) other traditional model performance metrics such as overall ROC-AUC and specificity, and (3) fairness measures, such as PPV differences across subgroups. We began prospective data collection in February 2025, and we will need AER and HIPAC support for future model implementation. Even in the absence of a formal trial, pain medicine clinicians may use the model’s predictions to inform care planning and patient optimization. In this case, we will also collect implementation metrics, such as the proportion of ML-identified high-risk patients referred to pain medicine. For the clinical trial of an AI-driven TPS, additional measurements will ideally need to be collected to evaluate success of the AI. The primary endpoint will be a reduction in incidence of post-surgical pain at 90 days by at least 10% among patients enrolled in the TPS compared to those receiving standard of care. Secondary outcomes include (1) patient-reported pain levels, (2) functional assessments, (3) opioid use (in MMEs), (4) healthcare costs at 30-, 60-, and 90-days post-discharge, and (5) clinician usage and feedback of the model’s APeX interface.

Section 7. Describe your qualifications and commitment:

Dr. Andrew Bishara, the project lead, has received a research career development (NIH K23) award to develop and implement AI models that predict a range of perioperative outcomes, with current efforts for validation and implementation underway at UCSF. Dr. Bishara will commit 10% of his time to supporting the development and implementation of this AI algorithm with the Health AI and AER teams. Dr. Christopher Abrecht, the co-lead, works closely with UCSF neurosurgery, seeing patients alongside them at the Spine Center and launching a preliminary TPS to support their patients. Dr. Abrecht is also committed to advising the implementation of the machine-learning model and supporting the development of an AI-driven TPS.

1. The UCSF Health problem

Large language models (LLMs) have rapidly emerged as powerful tools in clinical medicine, offering advanced capabilities in information retrieval, decision support, and diagnostic reasoning. Many clinicians (including at UCSF) are already using publicly available LLMs (ChatGPT, Claude, OpenEvidence) for clinical queries. Beyond challenges related to security and patient privacy, because the LLM does not interface directly with patient data, clinicians cannot easily obtain fully contextualized, patient-specific responses (taking into account age, past medical history, medications, etc.). The lack of an integrated solution means clinicians must manually enter curated data into an LLM to achieve the best results – a process with significant error risk.

Moreover, ensuring the reliability of AI-generated responses to clinical queries is paramount: mechanisms that ensure transparency of information sources are critical as they allow for physician oversight. Clinical AI tools must provide evidence-backed answers with clear citations to ensure human-in-loop medical decision making. With the rapid adoption of LLMs, UCSF clinicians will soon require an AI tool that delivers trustworthy, patient-tailored evidence within their workflow. OpenEvidence is an AI tool designed and trained specifically to deliver evidence-based and is already used widely in the UCSF and broader medical community (reportedly used by over 20% of U.S. doctors)​.

2. How might AI help?

We see an opportunity to integrate OpenEvidence within UCSF’s clinical environment to enhance its utility. We propose a light integration via a SMART on FHIR application that can securely access and extract relevant structured patient data (e.g., demographics, active medications, comorbidities, allergies) from the EHR. By automatically incorporating such context, an in-EHR OpenEvidence tool could deliver more precise and personalized real-time clinical support, reducing clinician burden and minimizing the risk of context omissions. For example, when a hospitalist asks a question (such as, “What is the optimal anticoagulation strategy for a patient with a DVT?”), the system would access structured data like the patient’s demographics (age, sex), active problem list (such as history of GI bleed), current medications, lab results, and other key context via the FHIR API. Using these details, the AI can interpret the question in light of the specific patient (e.g. “78-year-old female with atrial fibrillation, chronic kidney disease stage 3, on metformin…”). This context inclusion helps the LLM formulate an answer that is relevant to the individual patient, rather than a one-size-fits-all response and reduces reliance on the clinician to know every covariate that would change or contextualize the specific question at hand.

For this pilot, the app will pull key structured fields (patient demographics, medication list, allergy list, active diagnoses, and possibly recent vital signs or lab values) but not free-text notes, to limit complexity. With these inputs, OpenEvidence’s backend LLM can retrieve matching evidence (guidelines, clinical trial data, etc.) and generate an evidence-based answer. Crucially, the output will include transparent citations. By having the specific patient context, the AI can filter out irrelevant information and highlight applicable evidence. We believe this integration would produce outputs that are highly specific and grounded in evidence, closing the gap between raw EHR data and medical knowledge at the bedside.

3. How would an end-user find and use it?

Our primary end-users during this pilot will be inpatient hospitalists – specifically those on the UCSF Goldman(attending only service) service and the Medicine Teaching service (attending physicians and residents on the general medicine wards). We plan to deploy OpenEvidence as an embedded SMART on FHIR app, eitherwithin Epic or externally depending on UCSF AI teams recommendations (The OpenEvidence engineering team are willing to facilitate either). The interface would include a list of all patients within a user’s service list and allow selection of one patient at a time. Once launched, the user interface would be straightforward and familiar. The clinician is presented with a text input field labeled something like “Ask a clinical question about this patient.” They might type, for instance, “Does this patient’s diabetes and heart failure change the target hemoglobin for transfusion?” and hit submit (Figure 1; Section 4 below). The answer would consist of a few paragraphs of explanation, written in concise clinical language, and will include small reference superscripts or bracketed numbers corresponding to citations. The UI will prominently show these citations (e.g. a sidebar or footnote list of reference titles), as well as the ability to mark accuracy of both answer and provided citations.

 4. Concept figure

5. What are the risks of AI errors?

No AI is perfect – especially in high-stakes clinical settings – so we have identified several risks and failure modes, along with mitigation strategies for each:

• Overfitting to unique patient context: Evidence from the published literature is limited and often much simpler than highly complex clinical practice. If fed full patient chart information, the LLM might give an answer that overemphasizes those idiosyncratic details (e.g. focusing on a rare comorbidity that isn’t relevant to the question). This could lead to answers that are too narrow or not generalizable. Mitigation: We will work with the OpenEvidence team during initial testing to make sure it uses patient context appropriately. If a pattern of over-narrow answers is seen, we can adjust the algorithm to find a more appropriate balance.
• Incorrect answers or reasoning errors: The LLM could get the answer wrong, or even if an answer is correct, it could be derived from a suboptimal citation. It’s possible that integrating complex patient data may lead to either inaccuracies or suboptimal citation retrieval compared to what OpenEvidence is currently trained to do. Mitigation: We plan a rigorous accuracy review process. In the pilot, every answer the system gives will be logged, and a sample will be reviewed by a team of UCSF hospitalists for clinical correctness. We will track the percentage of answers with significant errors. If above an acceptable threshold, we will iterate on the model (for instance, by providing more training examples or adding rule-based checks for certain high-risk questions). Users will also have an easy way to report a “bad answer” from within the UI, which will alert our team to review that case. Furthermore, the tool will carry a disclaimer reinforcing that it is an assistant – final decisions rest on the clinician. Encouraging users to verify critical suggestions via the citations is another safety check (and one of the reasons we insist on transparent sources).

6. How will we measure success?

We will define success using both quantitative metrics from system logs and qualitative/clinical outcomes. Our evaluation will include:

1. Metrics from existing APeX data: Because the SMART app is launched within Epic (APeX), we can leverage backend analytics to capture usage patterns. Key metrics will include: the number of times the OpenEvidence app is opened (overall and per user per shift), the timing and duration of use, and the number of questions posed. We will also track how often users click on citation links to view more details – a proxy for how much they trust but verify the content. If possible, we will also examine workflow integration metrics such as: do clinicians tend to open the app on certain patients more (perhaps sicker patients or those on teaching teams) and at what points in the day (e.g. spikes during morning rounds)? A successful integration would show consistent usage (e.g. most hospitalists using it daily) and repeat usage by the same clinicians, indicating they found it valuable enough to come back.

B. Additional ideal measurements: We will complement the usage data with direct user feedback and clinical outcome assessments. This will involve:

• User surveys and interviews: We will administer brief surveys to the hospitalists and residents after they’ve used the tool for a few weeks. These surveys will gauge perceived impact on decision-making confidence, time saved, and trust in the AI’s recommendations.
• Comparative analysis (control groups): Ideally, we will design the pilot to include a comparison between different modes of information access:
• OpenEvidence with EHR context – the full integrated tool as described.
• OpenEvidence without EHR access – e.g. using the OpenEvidence app or web without patient context (so the clinician can ask questions but must manually input any patient details).
• UpToDate (or usual care) – clinicians relying on standard resources like UpToDate, guidelines, or consults, but not using OpenEvidence at all.

We can assign different ward teams or time blocks to different strategies and examine differences. Success would be reflected if the teams with EHR-integrated OpenEvidence report better qualitative outcomes than either control group.
 

7. Describe your qualifications and commitment

This project will be led by a UCSF faculty hospitalist with significant clinical and operational experience. Dr. Peter Barish is an Associate Clinical Professor of Medicine and is the Medicine Clerkship and Medicine Acting Internship director at UCSF St Mary’s Hospital. Dr. Barish has a long-standing interest in the field of clinical excellence and co-leads a biweekly DHM conference “Cases and Conundrums” as well as publishes a monthly newsletter (The CKC Dispatch) on current Hospital Medicine literature. He is also the UCSF co-site-lead of the national research study Achieving Diagnostic Excellence Through Prevention and Teamwork (ADEPT) where he and colleagues in DHM are currently investigating the use of LLMs to better understand and prevent diagnostic error.

He will be supported by Dr. Travis Zack, who is an assistant professor in the Division of Hematology/Oncology, an affiliate in the Department of Clinical Informatics and Technology (DoC-IT), the Computational Precision Health Program (CPH) and the Senior Medical Advisor for OpenEvidence. Additionally we have had discussions with OpenEvidence cofounder Zack Ziegler, who has enthusiastically expressed support for this project and will provide the engineers and technical expertise required to build the SMART on FHIR application here at UCSF (this would be the first integration of OpenEvidence within an EHR).