Fidelity of delivery and contextual factors influencing children's level of engagement: Process evaluation of the online remote behavioral intervention for tics trial

Abstract

Background: The Online Remote Behavioral Intervention for Tics (ORBIT) study was a multicenter randomized controlled trial of a complex intervention that consisted of a web-based behavioral intervention for children and young people with tic disorders. In the first part of a two-stage process evaluation, we conducted a mixed methods study exploring the reach, dose, and fidelity of the intervention and contextual factors influencing engagement. Objective: This study aims to explore the fidelity of delivery and contextual factors underpinning the ORBIT trial. Methods: Baseline study data and intervention usage metrics from participants in the intervention arm were used as quantitative implementation data (N=112). The experiences of being in the intervention were explored through semistructured interviews with children (n=20) and parent participants (n=20), therapists (n=4), and referring clinicians (n=6). A principal component analysis was used to create a comprehensive, composite measure of children and young people’s engagement with the intervention. Engagement factor scores reflected relative uptake as assessed by a range of usage indices, including chapters accessed, number of pages visited, and number of log-ins. The engagement factor score was used as the dependent variable in a multiple linear regression analysis with various contextual variables as independent variables to assess if there were any significant predictors of engagement. Results: The intervention was implemented with high fidelity, and participants deemed the intervention acceptable and satisfactory. The engagement was high, with child participants completing an average of 7.5 of 10 (SD 2.7) chapters, and 88.4% (99/112) of participants completed the minimum of the first four chapters—the predefined threshold effective dose. Compared with the total population of children with tic disorders, participants in the sample tended to have more educated parents and lived in more economically advantaged areas; however, socioeconomic factors were not related to engagement factor scores. Factors associated with higher engagement factor scores included participants enrolled at the London site versus the Nottingham site (P=.01), self-referred versus clinic referred (P=.04), higher parental engagement as evidenced by the number of parental chapters completed (n=111; ρ=0.73; P<.001), and more therapist time for parents (n=111; ρ=0.46; P<.001). A multiple linear regression indicated that parents’ chapter completion (β=.69; t110=10.18; P<.001) and therapist time for parents (β=.19; t110=2.95; P=.004) were the only significant independent predictors of child engagement factor scores. Conclusions: Overall, the intervention had high fidelity of delivery and was evaluated positively by participants, although reach may have been constrained by the nature of the randomized controlled trial. Parental engagement and therapist time for parents were strong predictors of intervention implementation, which has important implications for designing and implementing digital therapeutic interventions in child and adolescent mental health services.