Development and validation of a scale to evaluate treatment progress in secure mental health settings

Abstract

Introduction: Forensic Mental Health provision in secure healthcare is complex. The availability of a short and quantifiable, yet comprehensive instrument for summarising patient progress encompassing multi-professional clinical input would facilitate clinical decision-making. Objectives: To develop and validate a Progress Rating Scale (PRS) for use in secure healthcare to assess patient course in treatment. Aims: PRS items will reflect multi-professional clinical input. Measurement will be valid and reliable. Method: Scale items Development was undertaken at the Personality Disorder Service, Arnold Lodge Regional Secure Unit. Thematic analysis of 5 randomly selected archived treatment reviews resulted in a preliminary list of items. This scale was then piloted and refined via independent rating of further anonymised reports. Validation and further development To assess content validity, 3 independent raters applied the scale on archived treatment reviews of 12 randomly selected patients. Following examination of single-rating intra-class correlations (ICCs), items were revised to achieve greater content validity. Results: The domains of the scale were: Engagement, Behaviour, Mental State, Interactions with Peers and Staff, Insight, Supportive relationship, Employment, Leave, Violence/risk, Psychometric score and Final outcome (upon discharge). Initial inter-rater agreement ranged from fair to substantial (ICCs: 0.37-0.82). Following revisions, agreement improved ranging from moderate to substantial (ICCs: 0.63-0.92), the latter applying to most items. Conclusions: A scale for evaluating patient progress was developed based on multi-professional clinical input. The scale was refined to improve content validity. Internal consistency and factorial structure are under scrutiny and results will be available at the conference.