Quantifying white matter lesions in multiple sclerosis: A multiple technique comparison

Abstract

Background: Measurement of total white matter (WM) lesion volume is important for the treatment and diagnosis of multiple sclerosis (MS). Manual delineation is time consuming and susceptible to operator dependent variability. Semi and fully automated methods have been developed and widely used in research, but direct comparisons are limited. Objective(s): To quantify variability across WM lesion volumes from two semi-automated software packages JIM 7.0 (Xinapse Systems, Northants, UK) and 3D Slicer, and one fully-automated FDA-cleared and CE-marked method QyScore Methods: Total lesion volume was calculated for 44 MS research patients (mean age=53 (range 36-65), 16M/28F, 16 Primary Progressive/28 Secondary Progressive) using JIM, 3D Slicer, and QyScore Comparisons were performed by calculating linear regression with the agreement between the different software packages assessed using the Bland-Altman method. Visual assessment of the results from a subset of the cases was conducted by experienced image analysts to identify sources of discrepancy with neuroradiologist review pending. Result(s): Mean (sd) lesion volumes were 11.38 (8.00), 18.39 (12.65), and 26.38 (18.06) ml for 3D Slicer, QyScore and JIM respectively. Correlation coefficients were calculated with greater similarity found between 3D Slicer and QyScore in determining relative lesion volume compared to JIM. Bland-Altman analysis indicated significant discrepancy between all three methods with a percentage bias of +38% (167% CI) between JIM and QyScore, -46% (78% CI) between 3D Slicer and QyScore, and +79% (155% CI) between JIM and 3D Slicer. The difference between regression and bias results highlights the challenges in delineating WM lesions across a typical pathological range. Visual assessment suggests this is largely driven by erroneous grey matter inclusion using JIM and under sampling of lesions with 3D Slicer. In the most discrepant cases QyScore produced the most representative WM segmentations. An additional consideration with semi-automated software is user dependency. Average user-input time (minutes) was 30 for both JIM and 3D Slicer. Conclusion(s): The method used for the quantification of WM lesions significantly impacts lesion volumes and ongoing work to better characterize this variability is key for precision and efficacy in MS clinical decision making. Metrics evaluated by QyScore are fast, accurate and reproducible.