gms | German Medical Science

21. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin e. V.

Deutsches Netzwerk Evidenzbasierte Medizin e. V.

13. - 15.02.2020, Basel, Schweiz

Design, Analysis and Reporting of Multi-Arm Trials and Strategies to Address Multiple Testing

Meeting Abstract

  • Dmitry Gryaznov - University Hospital Basel, Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, Basel, Schweiz
  • Ayodele Odutayo - Li Ka Shing Knowledge Institute of St Michael’s Hospital, Applied Health Research Centre, Kanada; University of Oxford, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien
  • Bethan Copsey - University of Oxford, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien
  • Paul Monk - University of Oxford, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien
  • Benjamin Speich - University Hospital Basel, Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, Basel, Schweiz; University of Oxford, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien
  • Corran Roberts - University of Oxford, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien
  • Karan Vadher - University of Oxford, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien
  • Peter Dutton - University of Oxford, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien
  • Sally Hopewell - University of Oxford, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien
  • Matthias Briel - University Hospital Basel, Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, Basel, Schweiz
  • Douglas G. Altman - University of Oxford, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Oxford, Großbritannien

Nützliche patientenrelevante Forschung. 21. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin. Basel, Schweiz, 13.-15.02.2020. Düsseldorf: German Medical Science GMS Publishing House; 2020. Doc20ebmPP10-07

doi: 10.3205/20ebm056, urn:nbn:de:0183-20ebm0564

Published: February 12, 2020

© 2020 Gryaznov et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at http://creativecommons.org/licenses/by/4.0/.


Outline

Text

Background/research question: It is unclear how multiple treatment comparisons are managed in the analysis of multi-arm trials. We investigated strategies for managing multiple testing related to primary outcomes in multi-arm trials.

Methods: We investigated clinical trial protocols approved by ethics committees in the United Kingdom, Switzerland, Germany, and Canada in 2012 and their corresponding publications. We created a decision tool to determine the need for multiple testing procedures (MTPs) and compared the results of the decision tool to the analysis plan in the protocols. Pre-specified analysis plans in trial protocols were compared to those in corresponding publications.

Results: Sixty-four protocols for multi-arm trials were identified, of which 50 involved multiple testing. Nine of 50 trials (18%) used a single-step MTP and 17 (38%) used an ordered sequence of primary comparisons to control the overall type I error. In the 9 trial protocols that used a single-step MTP, 6 (67%) considered an adjustment in their sample size calculation to maintain statistical power and prevent type II error. Based on our decision tool, 45 of 50 protocols (90%) required use of a MTP but only 28 of the 45 (62%) accounted for multiplicity in their analysis or provided a rationale if no MTP was used. The remaining 5 of 50 (10%) protocols did not require MTPs based on our decision tool. There was little difference between industry and non-industry funded trials regarding the use of MTPs when required (Risk Ratio 1.12, 95% CI 0.57-2.22). We identified 30 protocol-publication pairs, of which 20 planned a MTP in the protocol. Four of these 20 trials (20%) did not perform the MTP in the publication and provided no rationale.

Conclusion: Strategies to reduce type I and type II errors were inconsistently employed in multi-arm trials. Selective reporting of analyses occurred in publications of multi-arm trials.

Competing interests: The authors declare no conflict of interest