Proportionate methods for evaluating a simple digital mental health tool

Abstract

BACKGROUND: Traditional evaluation methods are not keeping pace with rapid developments in mobile health. More flexible methodologies are needed to evaluate mHealth technologies, particularly simple, self-help tools. One approach is to combine a variety of methods and data to build a comprehensive picture of how a technology is used and its impact on users. OBJECTIVE: This paper aims to demonstrate how analytical data and user feedback can be triangulated to provide a proportionate and practical approach to the evaluation of a mental well-being smartphone app (In Hand). METHODS: A three-part process was used to collect data: (1) app analytics; (2) an online user survey and (3) interviews with users. FINDINGS: Analytics showed that >50% of user sessions counted as 'meaningful engagement'. User survey findings (n=108) revealed that In Hand was perceived to be helpful on several dimensions of mental well-being. Interviews (n=8) provided insight into how these self-reported positive effects were understood by users. CONCLUSIONS: This evaluation demonstrates how different methods can be combined to complete a real world, naturalistic evaluation of a self-help digital tool and provide insights into how and why an app is used and its impact on users' well-being. CLINICAL IMPLICATIONS: This triangulation approach to evaluation provides insight into how well-being apps are used and their perceived impact on users' mental well-being. This approach is useful for mental healthcare professionals and commissioners who wish to recommend simple digital tools to their patients and evaluate their uptake, use and benefits.