Reliability of digital ulcer definitions as proposed by the UK Scleroderma Study Group: A challenge for clinical trial design

Michael Hughes, Andrew Tracey, Monica Bhushan, Kuntal Chakravarty, Christopher P Denton, Shirish Dubey, Serena Guiducci, Lindsay Muir, Voon Ong, Louise Parker, John D Pauling, Athiveeraramapandian Prabu, Christine Rogers, Christopher Roberts, Ariane L Herrick

Research output: Contribution to journalArticlepeer-review

27 Citations (SciVal)
85 Downloads (Pure)


Introduction: The reliability of clinician grading of systemic sclerosis-related digital ulcers has been reported to be poor to moderate at best, which has important implications for clinical trial design. The aim of this study was to examine the reliability of new proposed UK Scleroderma Study Group digital ulcer definitions among UK clinicians with an interest in systemic sclerosis.

Methods: Raters graded (through a custom-built interface) 90 images (80 unique and 10 repeat) of a range of digital lesions collected from patients with systemic sclerosis. Lesions were graded on an ordinal scale of severity: 'no ulcer', 'healed ulcer' or 'digital ulcer'.

Results: A total of 23 clinicians - 18 rheumatologists, 3 dermatologists, 1 hand surgeon and 1 specialist rheumatology nurse - completed the study. A total of 2070 (1840 unique + 230 repeat) image gradings were obtained. For intra-rater reliability, across all images, the overall weighted kappa coefficient was high (0.71) and was moderate (0.55) when averaged across individual raters. Overall inter-rater reliability was poor (0.15).

Conclusion: Although our proposed digital ulcer definitions had high intra-rater reliability, the overall inter-rater reliability was poor. Our study highlights the challenges of digital ulcer assessment by clinicians with an interest in systemic sclerosis and provides a number of useful insights for future clinical trial design. Further research is warranted to improve the reliability of digital ulcer definition/rating as an outcome measure in clinical trials, including examining the role for objective measurement techniques, and the development of digital ulcer patient-reported outcome measures.

Original languageEnglish
Pages (from-to)170-174
Number of pages5
JournalJournal of Scleroderma and Related Disorders
Issue number2
Early online date27 Mar 2018
Publication statusPublished - 1 Jun 2018


Dive into the research topics of 'Reliability of digital ulcer definitions as proposed by the UK Scleroderma Study Group: A challenge for clinical trial design'. Together they form a unique fingerprint.

Cite this