Artificial Intelligence / Machine Learning Demonstration Projects 2025

Crowdsourcing ideas to bring advances in data science, machine learning, and artificial intelligence into real-world clinical practice.

AI-Augmented Fall Prevention Tool for Nurses

Submitted by Maria Yefimova on April 3, 2025 - 9:31am
Proposal Status: 
Open

The UCSF Health problem

Falls among hospitalized adult patients are a significant patient safety concern, associated with higher morbidity, prolonged hospital stays, and approximately $62,000 in non-reimbursable costs per fall.1 While our fall rate is below the national benchmark, falls are considered a never event.2Moreover, they are a quality indicator under UCSF Quality and Safety True North Pillar. Nurses play a critical role in fall prevention3 and are the intended end-users of our proposed AI-Augmented Fall Prevention Tool.

UCSF Health currently employs an evidence-based Fall Prevention Program that incorporates Fall TIPS4, including risk assessment, care planning, and the delivery of tailored interventions. Nurses assess and document fall risk every shift using the STRATIFY tool5, which is based on 5 features: recent history of falls, and observations agitation, visual impairment, toileting, and mobility. However, STRATIFY uses a fixed threshold for identifying “high risk” patients, which limits its utility in rapidly changing clinical situations. Additionally, many of its components are already documented in other nursing assessments but are not easily integrated with other data.

A UCSF pilot in 2020 evaluated the Epic AI Fall Risk Prediction model6 with 38 variables, but it did not outperform STRATIFY and was not adopted into the workflow. It also lacked actionable recommendations for end-users, limiting its utility in patient care. Despite this, AI remains a promising avenue for integrating broader clinical features into fall risk prediction.

Major limitations to the current fall prevention approach are 1) the over-reliance on an overall risk score rather than attention to individual factors, 2) the high cognitive load to assess the contribution of the medical history and recent changes in status to inform tailored care planning, 3) and the burden of documentation that takes away from direct patient care.

How might AI help?

The primary goal of the proposed AI-Augmented Fall Prevention Tool is to reduce inpatient falls through timely and targeted prevention. By leveraging real-time clinical data, the tool will assist nurses with care planning by providing tailored, actionable recommendations. It will also reduce documentation burden by synthesizing existing information into a user-friendly format. The tool will incorporate both risk prediction and large language models and will be capable of reading from and writing to APeX. The tool will consist of a risk prediction and clinical decision support system with an integrated end-user feedback loop (Figure 1) These interdependent components will be built across three phases.

Phase 1: The Risk Prediction model will generate a dynamic profile that presents 1) a risk score and corresponding stratification (e.g., high, medium, low) and 2) the clinical features underlying fall risk. We will use APeX and Incident Report (IR) data to label cases. Fall events are documented in a separate IR database, but as of January 2024, they are also documented in APeX in the Post-Fall Flowsheet and Note. Since January 2022, there have been 1,148 documented falls.

Predictive features will be selected based on research evidence7,8,9 and their availability in APeX. Our prior work has shown that nurses already document STRATIFY elements in other daily flowsheets. In addition to pulling these data, we will include clinical, environmental, and time-dependent variables such as time of day, unit type, demographics, medical history, admission assessments, vital signs, medications, lab values, new diagnoses/procedures, and functional status. The model will automatically update the risk profile as new data become available.

We will begin by focusing on structured data from the Admission Navigator, Diagnoses, Problem List, Medication List/MAR, and Flowsheets. We will later explore the added value of unstructured data (e.g., gait assessments in Physical Therapy (PT) notes) and the technical costs associated with their inclusion.

 How would an end-user find and use it?

Phase 2: The Clinical Decision Support System (CDSS) will be embedded in the patient’s chart.Nurses would interact with it at the beginning of the shift and when a patient’s clinical status changes. The tool will visualize each patient’s fall risk, display trendlines, and highlight top contributors to any change. A clinical advisory will appear when a new high-risk patient is identified or when a patient’s risk sharply increases. It would recommend tailored interventions or adjustments to the current Falls Care Plan based on research evidence10,11. Recommendations will incorporate patient preferences and conditions (e.g., language, visual impairment) and be presented in natural, LLM-generated language to ensure clarity and increase nurses trust in the AI.

Phase 3 will include End-user Feedback and a Tracking Dashboard.  The nurse will be able to accept or reject the clinical advisory recommendation. If accepted, the AI tool would automatically update the Care Plan, reducing the need for manual documentation. It would also generate a reminder to document the performed intervention by the end of shift, if not done already. If the nurse rejects the advisory based on their clinical judgement, they can provide a rationale using a standardized list of options that reduce the cognitive burden. This input will support reinforcement learning with a "nurse in the loop" model. A Tracking Dashboard built in Tableau will display model and user metrics, monitored by the Falls Committee. This interprofessional committee, which reviews all fall incidents, has been actively involved in developing this proposal.

Key stakeholders — including Fall TIPS leaders, Falls Champions, and nurse informaticists — will co-design the CDSS interface to ensure usability, minimal clicks, and integration into workflows. We will apply the Consolidated Framework for Implementation Research12 and the NASSS model (Non-adoption, Abandonment, Scale-up, Spread, and Sustainability)13 to guide adoption and sustainability.

What are the risks of AI errors?

Primary risks include false positives and false negatives. False positives may flag too many patients as high risk and lead to alert fatigue from unnecessary clinical advisory alerts. False negatives may potentially increase falls due to missed opportunities for fall prevention. We plan to track model fit statistics to reduce these risks. Additionally, we will review clinical advisory rejections, which may suggest excessive false positives, and their rationale. We aim to mitigate these issues by optimizing the thresholds, refining the model, and engaging our stakeholders. Moreover, to reduce bias, we will plan to periodically recalibrate the model across age, race/ethnicity, and diagnosis groups.

How will we measure success?

This multi-phase project requires significant clinical and technical investment, but it has the potential to improve patient safety, documentation efficiency, and reduce EHR-related burnout — offsetting its costs. It may also be extended to other preventable harms and serve as a model for nursing-led AI innovation. Lastly, it would generate pilot data for a larger extramural grant.

Aim 1: Evaluate the risk prediction model fit and compare its ability to identify high risk patients to STRATIFY. Model metrics will include False Positive Rate, False Negative Rate, Positive Predictive Value, and falls occurring in patients classified as “low risk”. We will adopt the model if it performs as well or better than STRATIFY. If successful, we propose to replace STRATIFY with this AI tool and embed it in the current Fall TIPS program.

Aim 2: Evaluate and continuously refine the user experience with brief surveys and focus groups with nurses as well as general AI tool usage with APeX metadata. We will also use APeX metadata to measure the change in time spent on fall related documentation.

Aim 3: Evaluate the impact on the number of inpatient falls per 1,000 patient days, and the rate of falls with injury. These measures are available through the Zero Harms Dashboard. We will consider abandoning the project if there is a persistent increase in falls especially in patients that AI flagged as “low risk”, high advisory rejection, or low user satisfaction. The thresholds for these outcomes will be decided upon with the stakeholders.

  • A list of measurements using data that is already being collected in APeX: AI toolusage (views of the risk profile, clickthrough rate on advisory recommendations, advisories accepted and rejected, rationale for rejection), documentation (time spent on fall related documentation, time to the completed documentation after the advisory reminder).
  • A list of other measurements you might ideally have to evaluate success of the AI: nurse satisfaction and trust in the AI tool (focus groups), nurse agreement with AI-generated explanations (surveys/focus groups), use of fall prevention resources (e.g., safety attendants, PT consults), rates of falls and falls with injury, and potential cost savings.

Qualifications and commitment

The project leads are Maria Yefimova, PhD, RN (5% effort) and Sasha Binford, PhD, MS, RN, PHN, AGCNS-BC (5% effort). Both are faculty in the Department of Physiological Nursing at UCSF School of Nursing. Dr. Yefimova is the lead nurse scientist with UCSF Health. With her implementation science background and experience evaluating remote patient monitoring programs, she will oversee the development and evaluation. Dr. Binford is the Nursing Clinical Quality Specialist. She is the Fall Subject Matter Expert and has served as the tri-Chair of the Falls Committee. Previously, she has led the development of the UCSF AI delirium risk prediction model. She will contribute her expertise on the model development and validation and will liaison with clinical and operational stakeholders.

Falls Committee members that include Clinical Nurse Specialists (Melissa Lee MS, RN, PCCN, GCNS-BC, NEA-BC, chair), Nursing Quality (Meghan Sweis, MSN, RN, CNL, CPHQ), and Continuing Quality Improvement (Adam Cooper, DNP, RN, NPD-BC, EBP-C), as well as Nursing Informatics (Kay Burke, MBA, BSN, RN, NE-BC) will provide in-kind effort on model validation and evaluation.

References

  1. Dykes, P. C., Carroll, D. L., McColgan, K., Hurley, A. C., Lipsitz, S. R., & Bates, D. W. (2023). Cost of inpatient falls and cost-benefit analysis of implementation of an evidence-based fall prevention program. JAMA Health Forum, 4(1), e225125. https://doi.org/10.1001/jamahealthforum.2022.5125
  2. Centers for Medicare & Medicaid Services. (2006, May 18). Eliminating serious, preventable, and costly medical errors - never events.
  3. Horta, R. S. T. (2024). Falls prevention in older people and the role of nursing. British Journal of Community Nursing, 29(7), 335–339. https://doi.org/10.12968/bjcn.2024.0005
  4. Dykes, P. C., Carroll, D. L., Hurley, A. C., Lipsitz, S., Benoit, A., Chang, F., Meltzer, S., Tsurikova, R., Zuyov, L., & Middleton, B. (2010). Fall prevention in acute care hospitals: A randomized trial. JAMA, 304(17), 1912–1918. https://doi.org/10.1001/jama.2010.1567
  5. Oliver, D., Britton, M., Seed, P., Martin, F. C., & Hopper, A. H. (1997). Development and evaluation of evidence-based risk assessment tool (STRATIFY) to predict which elderly inpatients will fall: case-control and cohort studies. BMJ, 315(7115), 1049–1053. https://doi.org/10.1136/bmj.315.7115.1049
  6. Cognitive Computing Model Brief: Inpatient Risk of Falls Epic Systems Corporation.
  7. Appeadu, M. K., & Bordoni, B. (2023). Falls and fall prevention in older adults. In StatPearls. StatPearls Publishing. Retrieved April 1, 2025, from https://www.ncbi.nlm.nih.gov/books/NBK560761/
  8. Mao, A., Su, J., Ren, M., Chen, S., & Zhang, H. (2025). Risk prediction models for falls in hospitalized older patients: A systematic review and meta-analysis. BMC Geriatrics, 25, Article 29. https://doi.org/10.1186/s12877-025-05688-0
  9. Ghosh, M., O’Connell, B., Afrifa-Yamoah, E., Kitchen, S., & Coventry, L. (2022). A retrospective cohort study of factors associated with severity of falls in hospital patients. Scientific Reports, 12, 12266. https://doi.org/10.1038/s41598-022-16403-z​
  10. Pierre-Lallemand, W., Coughlin, V., Brown-Tammaro, G., & Williams, W. (2025). Nursing-led targeted strategies for preventing falls in older adults. Geriatric Nursing. https://doi.org/10.1016/j.gerinurse.2025.02.005
  11. Ojo, E. O., & Thiamwong, L. (2022). Effects of nurse-led fall prevention programs for older adults: A systematic review. Pacific Rim International Journal of Nursing Research, 26(3), 415–429. https://he02.tci-thaijo.org/index.php/PRIJNR/article/view/258061
  12. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. https://doi.org/10.1186/1748-5908-4-50
  13. Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A’Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017). Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. Journal of Medical Internet Research, 19(11), e367. https://doi.org/10.2196/jmir.8775
Supporting Documents: 
PDF icon AI-augmented Fall Prevention Letter of Support

Comments

Mark Pletcher - April 12, 2025 - 9:01am

I like the idea.  It seems like there might be some risk that the model you fit from our data will not outperform STRATIFY or the Epic tool.  Can you say more about why you think your model will work better than these other tools?  And explain why you would implement this even if it did not outperform STRATIFY?

Maria Yefimova - April 18, 2025 - 12:58pm

Mark, thank you for your thoughtful comment!

STRATIFY was originally developed in 1997 and has known limitations – as do the other validated bedside nursing fall risk assessment tools. It’s intentionally brief and easy to use for a busy nurse but it's not very sensitive or specific (reported model statistics below 60% on a 2019 systematic review (1)). It includes only five general risk items – with a notable omission of high-risk medication – and doesn’t account for dynamic changes in patients' conditions during a hospital encounter. So lab values, vital signs, and other clinically-relevant risk factors do not get included in a patient's assessment for risk. Meanwhile, recent studies in 2020s (2-4) that use AI/ML have shown stronger performance, with AUROCs above 0.81. They pull in real-time data like vitals, medications, and nursing documentation – which is what we are also proposing. We believe the Epic fall model underperforms at UCSF because it wasn’t built with our local documentation practices in mind and doe not allow for modification to include additional factors. For example, it doesn’t include things like delirium screening that our nurses document routinely and is considered a significant risk factor for a fall.

Even if our model doesn’t outperform STRATIFY in predicting falls (but is equivalent), it would still add value. Instead of just producing an overall score or a score stratification, our tool will display what a respective patient is most at risk for with decision-support for nurses to target their interventions to – these actionable recommendations would be based on the dynamic changes. It will also automatically update the Fall Prevention Care Plan, which means less repetitive documentation and better tailored interventions. That’s something STRATIFY and the Epic model don’t offer, and we believe it could make our fall prevention efforts more efficient.

While we acknowledge that developing AI tools takes time and effort, we see this project having multi-pronged ROI. If we save even 10 seconds of documentation time per patient per shift across our average 500 adult inpatient census, that adds up to over $100,000 in nursing time each year. Beyond that, this is a real opportunity to build nursing capacity in AI development. This would be one of the first projects at UCSF – to our knowledge – where frontline nurses would lead the development of AI to optimize their own practice and improve patient outcomes.

 

References: 

1. Oliver, D., Britton, M., Seed, P., Martin, F. C., & Hopper, A. H. (1997). Development and evaluation of evidence based risk assessment tool (STRATIFY) to predict which elderly inpatients will fall: case-control and cohort studies. Age and Ageing, 26(5), 353–360. https://doi.org/10.1093/ageing/26.5.353

2. Tseng, C.-Y., Wang, Y.-H., & Lin, C.-H. (2023). Development and validation of a machine learning-based fall-related injury risk prediction model using nationwide claims database in Korean community-dwelling older population. BMC Geriatrics, 23, Article 830. https://doi.org/10.1186/s12877-023-04523-8

3. Deng, Z., Liu, S., & Wang, Y. (2022). A machine learning-based fall risk assessment model for inpatients. Scientific Reports, 12, Article 21640. https://doi.org/10.1038/s41598-022-21640-7

4. Cho, I., Park, H., & Lee, H. (2021). Clinical impact of an analytic tool for predicting the fall risk in inpatients: Controlled interrupted time series. JMIR Medical Informatics, 9(11), e26456. https://doi.org/10.2196/26456

Maria Yefimova - April 18, 2025 - 1:06pm

Forgot one more citation: 

Oliver, D., Papaioannou, A., Giangregorio, L., Thabane, L., Reizgys, K., & Foster, G. (2008). A systematic review and meta-analysis of studies using the STRATIFY tool for prediction of falls in hospital patients: How well does it work? Age and Ageing, 37(6), 621–627. https://doi.org/10.1093/ageing/afn203

Marilyn Bazinski - April 16, 2025 - 11:19am

Falls prevention is an extremely important topic. I think this is proposal is very comprehensive. It is exciting to consider the use of AI to the benefit of patients and bedside clinicians. 

Maria Yefimova - April 18, 2025 - 1:00pm

Marilyn, thank you so much for your support. We completely agree — falls prevention remains a critical priority, and we're excited about the potential for AI to support both patient safety and reduce burden on bedside nurses.

Lindsay Bolt - April 16, 2025 - 5:00pm

Using AI to pull additional relevant details from the record to supplement the STRATIFY score is promising. We currently do not have processes to help highlight patients that have had an improvement in their mentation (judgement, need for reminders about getting up with out calling first), nor to alert to those that are showing subtle changes (worsening NuDesc or other delirium screening). Would appreciate a tool that can present back to the clinician what they already know about their patient - but packaged to assist with clinical judgement for risk prevention interventions and prioritizing responsiveness to a patient. Thank you for working on this and trying to leverage AI to improve upon the scales we use to further describe and refine the patients that need the most attention and support.

Maria Yefimova - April 18, 2025 - 1:07pm

Lindsay, you’ve captured exactly what we’re aiming for! We want AI not to replace nursing judgment, but to amplify it. It would highlight relevant patterns and subtle changes that busy nurses might otherwise miss. Thanks for your thoughts, especially around mentation and delirium! We hope this tool can have meaningful impact on prioritizing care and tailoring interventions.

Meghan Sweis - April 16, 2025 - 8:03pm

There are several powerful aspects to this proposal. Tailoring interventions to fall-risk factors is a predominent "new" evidence based practice for both fall and injury prevention, yet many of healthcare's prediction scales and enculturated practices focus on "high risk versus low risk" and all-or-nothing fall prevention bundles (signage, wristbands, bed exit alarms). Utilizing AI and machine learning to develop a screening tool that also highlights specific risk factors and nuanced changes that may be missed by human cognition during a busy shift is a key step to support practice change. In addition, the ability for AI to present complex information in orgnaized, easy to digest, formats will limit the cognitive burden required to process many disparate information sources, allowing RNs to shift their time and expertise to engaging the patient and family in their specific plan of care.

One of the great aspects of AI is to quickly refine presented information based on end-user feedback. It may be beneficial to explore how the clinical advisory recommendations may be presented in a chat-bot style, where the end user could chat back to provide additional patient context/preferences etc and recieve updated recommendations related to the present risk factors. 

Maria Yefimova - April 18, 2025 - 1:12pm

Meghan we strongly agree with your thinking on the shift from binary risk classification to a more nuanced, risk-factor-specific focus. We hope both the risk prediction model and the clinical advisory powered by LLM would generate for nurses.

We're especially excited about reducing cognitive load and helping nurses focus more on meaningful interventions. The idea of a chatbot is very cutting-edge! I don't know if UCSF/APEX has this functionality but it would be powerful way to close the loop with the nurse. Thank you for your thoughtful suggestions!

Melissa Lee - April 17, 2025 - 8:02pm

I am excited to support the AI-Augmented Fall Prevention Tool proposal, which has the potential to enhance nursing practice at UCSF Health. By using AI to dynamically assess fall risk with real-time clinical data, this tool will help nurses make timely, informed decisions to prevent falls and associated harm. It integrates multifactorial risk factors, reducing the cognitive load on nurses to optimize patient care delivery.

The Clinical Decision Support System will provide actionable recommendations based on current data, leading to a proactive approach to fall prevention. This project aligns with our commitment to patient safety and high-quality care, offering a promising solution to reduce falls and improve outcomes. By synthesizing existing information into a clinician-friendly format, the tool could not only enhance the accuracy of risk assessment but also streamline documentation processes, allowing nurses to focus more on direct patient care. It will empower nurses with advanced tools and reduce EHR-related burnout, making it a meaningful advancement in our fall prevention efforts.

Maria Yefimova - April 18, 2025 - 3:32pm

Thanks, Melissa! We, as nurses, are driven by our committment to patient safety and that is why we focused on such an important issue as inpatient falls. With the ongoing burden of documentation, this AI tool would empower nurses to shift their time to taking care of patients -- improving both health outcomes AND nursing satisfaction with their work environment (another alignment with "Our People" True North Pillar!)

Vivian Huang - April 18, 2025 - 1:25pm

What an exciting proposal! Would love to see how AI can support staff's clinical judgement in piecing together the full patient picture and fall risk. It can greatly impact multiple TN pillars and strategie priorities - zero harm, patient and staff experience, financial strength. I also foresee it paving the way for how AI-integrated clinical decision support systems can support other initiatives across the health system.

Maria Yefimova - April 18, 2025 - 3:34pm

Totally agree, Vivian! We are excited that UCSF Health would consider investing in building nursing capacity for AI that can be applies to other pressing clinical issues.

Lauren Frahm - April 18, 2025 - 2:00pm

This proposed use of AI is innovative and relevant- helping nurses seamlessly integrate evidence into their practice and individualize care for patients to prevent falls. They have clearly considered the needs of the nurse end-users as well as the patients in this proposal.  

Maria Yefimova - April 18, 2025 - 3:37pm

Lauren, we are excited to engage our frontline nurses, through Falls Committee and our Nursing Shared Governance structure, as co-designers in this project. Nursing representation is oftentimes missing in those crucial stages and can negatively impact the resulting end-user experience and adoption in the clinical setting. 

Esther Rov-Ikpah - April 18, 2025 - 3:59pm

I love this proposal and idea. I think that leveraging the health record in a meaningful that is updated with the patient's clinical condition in a more dynamic way is an exciting thing to consider and determine if it can be meaningful to help prevent falls. I'm excited ot hear what comes next. 

Marianne Biangone - April 21, 2025 - 1:33pm

There are several compelling elements to this proposal. Shifting from generic, one-size-fits-all approaches to more tailored, data-informed interventions reflects a meaningful evolution in clinical practice. Historically, many systems rely on broad categorizations or static workflows that often overlook the nuance needed to address individual patient needs. Leveraging AI and machine learning to surface context-specific insights—especially those that may go unnoticed in the fast pace of clinical care is a crucial step toward improving outcomes and workflow efficiency.

Moreover, the capacity of AI to synthesize complex data into streamlined, actionable formats has the potential to reduce cognitive load, enabling nurses and other clinicians to focus their time on what matters most: meaningful engagement with patients and families to co-create care plans that reflect shared goals and values.

Importantly, the success of this project lies not only in its application to falls prevention, but in its broader potential to support a wide range of clinical protocols. For example, sepsis detection protocols already rely heavily on real-time data pulls to alert providers of the need for urgent intervention. This same model could be extended to other areas, such as pressure injury prevention, early deterioration alerts, or even discharge readiness, each benefiting from AI’s ability to detect subtle patterns and changes that may otherwise be missed.

An exciting future opportunity may lie in enabling real-time interaction with the AI-generated recommendations. A conversational interface, such as a chatbot-style advisory tool could invite clinicians to contribute additional context or patient preferences, in turn receiving refined, individualized recommendations that dynamically evolve with the clinical picture.