CTSI Annual Pilot Awards to Improve the Conduct of Research

An Open Proposal Opportunity

Internet Based Solution for Enhancing Patient Recruitment

Proposal Status: 


Research that requires real-time recruitment in emergency department/acute care setting is labor intensive and costly. Relying on clinicians leads to low recruitment. Clinicians in busy acute care settings often find it challenging to screen and enroll patients for research studies while maintaining clinical priorities. This is further complicated by several different ongoing studies (often led by research faculty from outside departments) and it remains challenging to  keep clinicians current on the various eligibility criteria due to variable work hours.  While most emergency departments use the traditional approach of having a research assistant who screens for eligibility based on physician/resident notification, this approach has inherent problems: the cost and/or availability of research assistant coverage 24/7, variability among physicians in their notification of research assistants about eligible subjects, and the resulting patient ineligibility due to lack of timely screening. We believe that some of these drawbacks of the traditional approach could be overcome by use of an automated electronic screening tool.


We propose to examine the utility of an automated electronic clinical research screening technology to assess the comparative effectiveness of this method to the traditional system of research associates. This technology is currently installed at the University of California, San Francisco Medical Center Emergency Department (UCSF). We will utilize this technology in the setting of an ongoing NINDS clinical trial.


We propose to accomplish this in a pilot study by

(1) Demonstrating the available features of the automated tool using an ongoing clinical trial, the NINDS funded POINT trial.

(2) Comparing the performance characteristics of the automated tool to a traditional patient recruitment method for screening and enrollment.


Our specific aims are to:

Specific Aim 1: To identify and document study subject characteristics that will be used in developing a screening criteria for the automated tool (Phase 1)

Specific Aim 2: To demonstrate the performance characteristics and comparative effectiveness of automated real-time screening tool using the ongoing  clinical trial in the emergency department at UCSF (Phase 2)

Hypothesis: Automated screening will be more sensitive, and less specific than the traditional method of using research assistants in identification of subjects.

Specific Aim 3: To demonstrate that the operation of real-time clinical research eligibility screening meets HIPAA privacy requirements and does not compromise PHI.

Hypothesis: The technologies and architecture that is employed in developing the Patient Locate service will be sufficient to ensure that HIPAA privacy requirements can be met.

Study Design: In the first three months of the study, we will study the baseline characteristics and presenting symptoms of the eligible study subjects as well as patients enrolled in POINT. We will use the information to develop screening criteria which will be tested in the second phase of the study. During the second phase  KDH system will be activated to identify study eligible patients and the effectiveness of the tool will be compared with the traditional method of recruitment.


Metrics: Sensitivity, specificity, positive and negative predictive value, and likelihood ratios will be calculated to determine the screening performance of the automated screening tool system for eligibility. The performance characteristics will be compared to the research assistant supported system currently in use at UCSF. 95% confidence intervals will also be calculated to assess the strength of the point estimates.

Anticipated result and impact on trial enrollment procedures:

Upon the culmination of this study, the results will help our research team and the CTSI better understand the relative merits of the automated real-time patient recruiting system.

Budget and Justification: Salary support for the investigators and a research assistant for a 12 month period ($60,000)

Collaborators: John Stein (Co-PI), Department of Emergency Medicine, Claude Hemphill, Department of Neurology and Dan Carnese, KDH systems



I think this is a great idea. I've thought about this type of 'solution' as well, and one thing that you might want to look at is the 'fatigue' factor in clinical trials (and whether your solution will decrease recruitment fatigue). There's not much out there in the literature on site fatigue. When planning a multicenter trial this needs to be incorporated in the anticipated recruitment rate and decreasing fatigue would be an attribute that would increase the value of your product.

If/when you begin exploring such methods, I'd be happy to provide agent-based modeling and simulation feedback. Unfortunately, even though the benefits of such efforts can be great, a lot of work (and for some, a considerable learning curve) is required up front.

Thanks Anthony. I would like to learn more about the modeling and simulation feedback techniques and will contact you.

Thanks for the suggestion. The pilot will be single site study but the next phase will involve more than one center and we will study the "fatigue factor".

I like your idea. If you set up a fairly general process and show it's effective, it's likely that it could be rolled out to settings beyond the ED.

Is the patient recruitment tool already available at the UCSF ED? Could you provide more specifics regarding how it works? If a UCSF investigator wanted to collaborate, how would he/she input the specifics for their trial into the tool? What aspects of the tool do you propose to examine? Are you currently enrolling trials? A description of these trials might better explain how the tool works. How do you think the tool might be improved? What are the current challenges to HIPAA faced by the tool? Do you propose a method where patients can be shown recruitment flyers for current ongoing research at UCSF so they may contact researchers who are conducting trials where the patient may be qualified?

I will email my responses and will copy my collaborator John Stein who has been using this tool for his study.

outstanding idea!

Automated eligibility screening has been a "holy grail" for clinical research informatics for some time. There are a number of both academic and commercial systems for this. I assume this is the commercial KDH system that you propose to evaluate? Depending on the system particulars, and ancillary factors like workflow integration, the performance results may have limited generalizability. This evaluation of a commercial system could be a proposal to the mHealth/Digital Health RAP awards. http://accelerate.ucsf.edu/funding/rap

I think this is a great idea, especially for time-sensitive enrollment in the ED. There is considerable detail on the parameters that will be compared between the automated system and the RA-based system, but more detail on the actual pilot study would be helpful here to assess which eligibility criteria would be amenable to this system and which would not. Is the proposal to use this for a NETT trial? Which one?

Thanks Anthony. We will be using this for the POINT trial. The proposal has been revised to include information about the design and conduct of the pilot study.

This is of high interest to Med Center, SOM and researchers across UCSF. We plan on making investments in the coming year to ensure a realtime recruitment process and technologies are in place to improve just-in-time recruitment. Complemented with the new Recruitment Management System being developed, we envision this technical eco-system to be as advanced as UCSF"s standing. A study that would help shape these upcoming investments would be a welcome input and I think very worthwhile.

Fantastic idea, timely, and one with a tangible ROI.

This is a very timely proposal that will help us in making decisions on how to proceed to make real-time patient recruitment a reality for UCSF investigators.

Commenting is closed.