gms | German Medical Science

Introduction: Hazard ratios are almost exclusively used to communicate treatment effects in survival analysis. As results of Cox regression models, they are easy to implement and summarize the effect in a single number. However, the hazard ratio has also been criticized for a number of reasons in recent years. One criticism is the implied selection bias in randomized controlled trials (RCTs), which is an inherent consequence of the partial likelihood approach to parameter estimation. While the first factor in the partial likelihood is still characterized by a randomized comparison, all subsequent factors can potentially generate bias in case of large treatment effects and unobserved heterogeneity [1]. Theoretical considerations showed that estimated hazard ratios could be heavily biased in such situations [2]. However, there is no evidence of the practical relevance of this problem. The aim of the present work is thus to analyse the extent of this selection bias through an empirical study of 28 large RCTs and to derive appropriate recommendations for dealing with the interpretation of the hazard ratio in RCTs.

Method: For measuring selection bias, observations are successively deleted from the datasets which were sorted by observation time. Then, a Cox model is fitted for each reduced data set. In addition, the balance of two confounders (age and gender) is measured by z-differences [3]. This procedure of successive cancellation and application of a Cox model is repeated until the studies no longer contain a sufficient number of events. If there is unobserved heterogeneity in the studies, a systematic selection of patients is expected, which results in a bias in the hazard ratio. A small proof-of-concept simulation showed that this method could indeed identify such a selection bias. Finally, we ran these analyses in the individual patient data of a previously used sample of 28 RCTs [4].

Results: No systematic bias in the estimated treatment effect could be found for most of the studies. Apart from random fluctuations, the hazard ratios showed rather stable estimates in the course of deleting observations. The associated balance in the measured covariables age and gender also remained stable, indicating no imbalance of these confounders.

Discussion: The analyses showed that the criticism of hazard ratios as an effect measure for evaluating treatment effects is not justified. Selection bias due to unobserved heterogeneity have no practical relevance when evaluating survival data with the Cox model, which is partly due to the rare occurrence of large treatment effects in RCTs [5].

Conclusion: In summary, it was shown that the impressive warnings from various authors are of a theoretical nature and standard analyses with the Cox model are generally not distorted by the described effect. However, all other reported problems of hazard ratios in terms of interpretability or non-collapsibility stay virulent and should be taken serious by the biostatistical community.

The authors declare that they have no competing interests.

The authors declare that a positive ethics committee vote has been obtained.