CTSI Annual Pilot Awards to Improve the Conduct of Research

An Open Proposal Opportunity

Using Mobile Technology and Game Dynamics toRecruit and Retain Research Participants

Proposal Status: 

Rationale:  Enrolling subjects for longitudinal research studies, ensuring their compliance and retaining them over a period of time pose a significant challenge to researchers. As an example, we had 62% and 30% retention at years 1 and 3 in a recently concluded study on knee osteoarthritis. To overcome these challenges and to optimize workflow associated with data acquisition, quantification, we have developed a design which combines multiple computing platforms to create a seamless, user-friendly experience for all participants (researcher and subject).  The platforms [using  Ruby on Rails (RoR),  PostgreSQL, iOS  and representational state transfer (RESTful) API]  are designed to enable researchers and subjects to complete data acquisition forms and surveys on tablets which then get quantified and submitted to our cloud servers. Qualitative data show improved subject satisfaction compared to paper forms, reduced paper use by researchers and study coordinators, reduction in man-hours by researchers and study coordinators to manually quantify survey scores and input data into the database and better quality control.

Based on these initial experiences using mobile technology for enhancing subject’s participatory experience and streamlining associated workflow, we propose – (1) the expansion of our current platform to UCSF Imaging clinics (for recruitment and improving care experience), (2)  a one year research study on obese individuals examining effect of weight-loss on knee tissue health using quantitative imaging. This will enable us to analyze the efficacy of the platform towards maintaining compliance and increasing retention. (3) Additionally, we also intend to pilot a networked module incorporating social gaming and feedback principles to assess behavior modification in this population which is of special interest to our lab for our research on knee osteoarthritis prevention.


PLAN:  Recruitment component: (1) Over the 1st three months, enhance the current tablet computer applications to include    interactive versions of intake forms, standardized questionnaires, as well as, modules which simulate the procedure to be performed (MRI, CT, radiography) to address common issues like claustrophobia, restricting movement during scanning etc.. This module will also offer the patients to learn about unique capabilities of the UCSF Imaging centers (safety regulations, metal suppression sequences, quantitative imaging capabilities, other research studies which offer volunteer opportunities) along with a brief bios of the personnel who will be interacting with the patients during their procedures.  While the patients learn about the UCSF imaging capabilities, they will have the option to explore ongoing research participation opportunities  and, if interested, sign up to be contacted by a study coordinator. We have multiple longitudinal trials underway at the moment which will benefit from this mode of recruitment. Possibility of using mHealth frameworks implemented at UCSF will be explored to allow for future adaptability across other UCSF departments. (2) Over months 3-9, administer the tablets to all patients at the UCSF imaging centers (Orthopedic Institute, China Basin Landing and Mission Bay) scheduled for musculoskeletal imaging for clinical or research purposes.

Retention and gaming component: (1) Over months 4-12, twenty-five adult obese subjects from the UCSF- Weight Management Program will be recruited. These subjects will undergo MR imaging at baseline and 6 months, 9 months for quantification of knee tissue health using our standardized metrics. At each time point, subjects will complete surveys related to their levels of physical activity, nutritional data, and incidence of musculoskeletal pain/injuries every month. Additionally, the subjects will complete surveys  remotely using mobile applications (described later) every 6 weeks. (2) Over the 1st three months, develop mobile phone applications for surveys and a “gamified” feature where the research participants can track their scores (levels of physical activity, functional status, pain status, calorie intake) and compare it to other participants, promoting increased participation of physical activity using competitive motivation. A virtual character will be designed (eg: an alien trying to reach home planet) and the participant  will need to complete all study associated tasks, as well as, maintain a certain level of physical activity, consume the recommended diet to enable the character to succeed at the game. Periodically, information on the benefits of the interventions will be incorporated, feedback on the performance with resulting improvements will also be provided and quizzes on weight-management will be incorporated. These approaches have been shown to be effective in management of diabetes, improving physical activity in children and improving walking capacity in adults to cite a few (1-3).

Long term plan: This pilot will be used to demonstrate feasibility, efficacy and flexibility of the platform using UCSF Imaging clinics and research. In the next phase, an NIH application will be submitted for a larger, longer research project where the platform will be implement in obese individuals undergoing surgical and non-surgical weight-loss and followed-up for four years. Eventually, the platform will be made available to all UCSF researchers interested. Recruitment and clinic modules will be made available to other interested UCSF departments immediately. 

Criteria and metrics for success: (1) Percentage of participants enrolled in research studies from the UCSF imaging centers (currently unknown) (2) Clinic patients and research participants will be invited to complete an online survey assessing their satisfaction with their experience, likelihood to stay in the research study and likelihood to return to UCSF for future appointments. (3) Retention in the research studies over a one year period will be compared to retention in all of our other longitudinal studies. (4) Impact of weight-loss on changes in composition of knee articular and meniscal cartilage using quantitative imaging, intramuscular fat content of thigh muscles, levels of physical activity, amount of weight-loss will be assessed at one year.


Approximate Costs and Justification: Total Budget = $95,350; 2 Tablet computers for development @ $400 ea. = $800; 20 Tablet computers for deployment @ $400 ea.    = $8,000; 15% effort from the PD/PI  (Dr. Kumar)  = $8,550; 25% effort from QUIP-C software developer over the year = $25,000; Consultant fees for gamification of the applications =  $25,000; Partial costs for MR imaging of research participants (shared with other grants)  = $28,000


Collaborators:  Deepak Kumar (MQIR); Vivek Swarnakar (QUIP-C), Sharmila Majumdar( MQIR), MQIR), David Dean ( QUIP-C), Richard Souza (MQIR), TBD(Game developer)  



1)      Aoki N, Ohta S, Masuda H, Naito T, Sawai T, Nishida K, Okada T, Oishi M, Iwasawa Y, Toyomasu K, Hira K, Fukui T. Edutainment tools for initial education of type-1 diabetes mellitus: initial diabetes education with fun. Stud Health Technol Inform. 2004;107(Pt 2):855-9.     

2)      Southard DR, Southard BH. Promoting physical activity in children with MetaKenkoh. Clin Invest Med. 2006. 29(5):293-7.

3)      Consolvo S, Everitt K, Smith I, Landay JA. Design requirements for technologies that encourage physical activity. In: Grinter R, Rodden T, Aoki P, Cutrell E, Jeffries R, Olson G, editors. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; 2006 Apr 22-27; Montreal, Quebec, Canada. New York: ACM; 2006. p. 457-66.


I love the idea of implementing what we know about reward processing to encourage patients to participate in research studies... and mobile technology seems like the perfect place to make this a reality. I don't know who your target population is (seems potentially very diverse), but I think this approach would be extremely effective with adolescents (who I work with in school-based settings). Would be happy to collaborate if this population is of interest!

Hi Kaja, Thanks for the comment ! The adolescent population happens to be one of our target populations for an upcoming project. We are looking to implement similar concepts based on mobile technology, game dynamics and social media as a part of this community based project to help answer our research questions. I will get in touch with you to discuss potential common interests !

I really like the idea of having a patient-facing tablet or kiosk-based system that can collecting self-reported questionnaire data in the waiting room. I could see parts of this model as potentially generalizable to the ED or urgent care setting. I believe that Ralph Gonzales and his collaborators in the Screening and Acute Care Clinic has incorporated a self-report registration system that includes simple questions for UTI screening and treatment at triage, but the ability to link longitudinally for follow-up data collection could be very useful for clinical research.

Thank you for the comment Anthony. The concept is definitely generalizable to other settings and would eventually be. Since we have easiest access to Imaging center clients, we have proposed initial development and deployment on this sample. We already have a lot of our research questionnaires on tablet computers which quantify and communicate the data to our cloud servers with immediate access for researchers. I will look into what Ralph has at the Acute care clinic.

I love this idea. It would be a great fit for collaboration with the "Digital Clinical Research Center" (see other proposal) - could have gaming modules that encourage longitudinal participation in the dCRC generally, and others that could be easily adapted and plugged into individual projects using dCRC patients and resources. Are envisioning easy adaptability to other projects with a web-based presence?

Thanks Mark ! I saw your comment on the dCRC as well and I think collaborating would be perfect. Initial prototype development and deployment is to be done on Radiology patients and participants but yes, easy adaptability and providing the tools to the UCSF community is the key to the success of this proposal over long term.

I love the idea of using mobile technology and gamification to help streamline the questionnaire/intake form process, and enhance the overall research participant experience. I think ideas in this proposal are very relevant and have tremendous potential. To echo some of the other comments, I can see this being applied to many other areas of research and medicine. I am also very interested in the longitudinal component. I think it would be great to see what kind of long-term impact mobile technology and gamification can have (in your case: subject retention rates and compliance). Very cool!

Thanks you for the comment ! Gamification has shown tremendous potential as can seen by the success of ventures like healthTap and Keas. we hope to see similar efficacy in research !

Great idea. I feel there is a need for more transparency to help demystify clinical research to both patients and study participants – and this project definitely proposes to accomplish this. In addition to the interactive environment, I also think it may be helpful to embed other forms of media in the future, such as videos depicting imaging procedures and current research that is being done.

Thanks Justin ! Having videos would be great. UCSF-Radiology has a youtube channel which already has some videos for lay public to "demystify" some of the commonly used imaging techniques. We do envision using these modules as recruitment tools as well where the patients can opt to brows current volunteer opportunities as well as learn about on going research projects.

Interesting idea, that fits well in San Francisco. I think that there will be generational gaps, where certain age groups will have difficulty with the interaction and may not trust non-paper formats. For the younger population, this technology could be used across multiple environments - from the dentist to the ER. Within the research perspective, these technologies would hopefully improve retention through regular electronic feedback, reminders, and perhaps continued data collection after initial contact.

Like others, I think introducing new ways to educate and engage patients at point of care, to enhance both the care experience (and post care compliance) as well as likelihood of successful participation in research, is important. This kind of work though may be better designed if specific outcomes are identified and based on those, specific targets clarified for developing tools/apps/content. Having been involved in many experiments that use technology to change behavior, the ones that work are when specific needs are tied to clear content or tool delivery strategies. Currently this feels aspirational (great) but doesn't as yet feel like there's a workable plan with predicted impact on research. I realize that a page isn't much space, but if you can tie an assessment of outcomes on research to a specific tool/social media/content strategy, that will help the assessment of your proposal.

Like everyone else, love, love, love x 10 the concept! But also like Mini states below, I think this could be improved with more specificity and focus. 1. The application for ALL patients visiting the radiology centers is likely to be very popular, but this feels more like a clinical service/delivery application rather than one that is central to CTSI goals in training/research infrastructure/etc. If you are proposing to recruit patients to clinical trials through this mechanism, then that's a different story... and would seem fully in-scope. In which case, you should provide more details about how you propose to do this. 2. The research app is also very interesting. However, it is not clear what "gap" this tool will be addressing. If you could elaborate on what the current key deficiencies/limitations and barriers are with the current system, this can help drive the design and strengths of your propsoal. It can/should also drive the metrics you will use to measure the success of this innovation. For example, how big of a problem is study drop-out among participants visiting the radiology centers? 3. The gaming framework sounds very cool, but do you have a specific reason that you think this will improve some specific deficiency with the current system? Is there any previous experience or research in gaming you could point to that will help guide this specific application of gaming? 4. What is your business model for these tablets, particularly those for regular patients? Who will pay and why (after the research is gone)? 5. These will be heavily dependent on Med Center IT support... worth budgeting/partnering with them (if not already... not sure). 6. The longitudinal mobile app function for monitoring and feedback is a no-brainer. CRC or other relevant CTSI program should just start doing this with you now! 7. Why are you providing the tablets to only half of the sample? Again, great ideas and great area to apply IT solutions!

Great idea, I agree with Ralph that further defining the "gap" in the current system is needed and with Mini regarding targeting of specific outcomes. We have a three of these "kiosk" type apps deployed now and all of them have been very popular modes of collecting information. from an enterprise IT perspective, I am concerned about having these developed as "one-off" applications that don't interact with other UCSF systems or each other. We've painstakingly developed a framework to address interoperability while adhering to PHI and HIPAA requirements. Why re-invent the wheel? We're more than happy to share this framework for use by outside entities. My adivce is the sooner we standardize the sooner we can actually start building the type of eco-system we need to develop these applications as meaningful platforms for data collection and collaboraton.

For some reason the previous reply was posted as anonymous. I posted it.

Regardless of the proposal being funded, it would be great to know what frameworks exist on campus that would meet the needs identified in the proposal. Specifically a framework that in a simplified manner improves a patient's understanding of the complex medical procedure they would be undergoing, in this case radiological imaging and while doing this raising their awareness to ongoing clinical research options while they are visiting UCSF.

Here's a link that describes one such UCSF framework and some of the applications built on it. (http://www.ucsfmservices.org/home/) The UC MWF (Mobile Web Framework) is also in use here in the form of the UCSF Mobile app. When Tuhin Sinha was with the radiology team they had also created a very nice framework based on Ruby on Rails which is still in use. Each of the frameworks has their strengths. What we have found is that the design of the intervention and the creation of the content to be used by the apps are the tasks that take the longest not development.

Thank you for sharing the link. Some aspects of this proposal came from discussions I had with Iana Simeonov in the context of "Choose Your Poison" app. I believe it was built using the framework you referenced and we certainly plan on using the framework for this project. The simplicity and user-friendliness of that app makes it interesting as an approach to improve user participation and retention in clinical research. We wanted to also explore further improvements in recruitment while not incurring any additional marketing costs, hence the suggestion to try it in the clinical setting. The framework developed by Tuhin Sinha is quite actively used and we have further extended it on the data capture side. However the human interaction side of it needs further development, to better leverage mobile technologies.

I have similar thoughts as voiced by some of my colleagues previously. Great idea and great project, but I am missing/confused about some of the details of the research component. In terms of likelihood to stay in the longitudinal research project, I assume that this could be across a variety of possible projects, or are there 1 or 2 projects that you will be focusing on? You discuss game dynamics for a particular research project in terms of the participant's experience with being a subject in a research study. How is this going to happen? Are you and your collaborators going to develop this for each person's research project? I think I understand how the clinical intake forms could be translated to interactive versions on the tablet and that satisfaction with the tablets/process is one outcome.(Other outcomes could also be defined for this component of the work.) I am less clear about how the tablets are linked and integrated with other longitudinal research and who is doing the translation of that research (a lot of work) into the interactive module. Exciting ideas but some of this additional information would be quite useful.

Commenting is closed.