gms | German Medical Science

Background: Statistical methodology plays a crucial role in drug regulation. Decisions by the FDA or EMA are typically made based on multiple primary studies testing the same medical product, where the two-trials rule is the standard requirement, despite a number of shortcomings.

Methods: A new approach is proposed for this task based on the harmonic mean of the squared study-specific test statistics. Appropriate scaling ensures that, for any number of independent studies, the null distribution is a chi-squared distribution with one degree of freedom. This gives rise to a new method for combining one-sided p-values and calculating confidence intervals for the overall treatment effect.

Results: A comparison with the two-trials rule is made, as well as with alternative research synthesis methods. An attractive feature of the new approach is that a claim of success requires each study to be convincing on its own to a certain degree depending on the overall significance level and the number of studies. The new approach is motivated by and applied to data from five clinical trials investigating the effect of Carvedilol for the treatment of patients with moderate to severe heart failure.

Conclusions: The harmonic mean chi-squared test is an attractive alternative to the two-trials rule as it has more power at the same Type I error rate and avoids the evidence paradoxes that may occur close to the 0.025 threshold. It provides a principled extension to substantiate research findings from more than two trials, requesting each trial to be convincing on its own, and allows for weights.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.