gms | German Medical Science

Background: Causal inference from observational studies faces a wide variety of challenges in particular in a setting with a rare outcome event and a large set of potential confounding variables. The shift towards focusing on modelling the relationship between the treatment assignment and the covariates instead of their association with the outcome to adjust for the effects of selection bias presents a suitable bypass to the main issue at hand. The probability of an individual to receive the treatment given the patient's information, known as the propensity score, can be used in the process of matching or weighting the observational data to combat the inherent imbalance of a patient's baseline characteristics in observational studies. Alternative proposals to tackle the shortcomings of observational data are based on traditional outcome regression models, also in combination with inverse probability weighting. However, a major strength of the propensity score methods is supposed to be their handling of a large number of confounders and few observed outcomes due to the focus on the exposure-covariates associations as an intermediary step.

Methods: A study to estimate the marginal causal treatment effect of a computer tomography scan examination of patients undergoing coronary artery bypass surgery (CABG) on the postoperative stroke risk served both as a real-data example and as a motivation for a comparative simulation study. In the case study, covariates were classified, together with the principal investigator of the study, by assuming that they were causes of treatment (T), outcome (O), or both (TO). Consequently, we applied the disjunctive cause criterion [1] by selecting variables classified as T, O, or TO, and the original recommendation by Rosenbaum and Rubin [2] by selecting O and TO variables. In the simulation, we varied sample size, magnitude of true effect, and effect size of confounders, and some scenarios also assessed the consequences of unmeasured confounding. The behaviour of four well-known causal inference approaches (Propensity score [PS] matching, Inverse Probability Treatment Weighting [IPTW], G-computation and Double Robust G-computation) were compared. We also considered variants of these methods involving weight truncation, variable selection and shrinkage. The target estimand in the study was the average treatment effect, and performance was evaluated by comparing the root mean squared error.

Results: Even though propensity score analyses (PS matching and IPTW) circumvent problems of estimating a multivariable outcome model with too many variables and few events, both PS matching and IPTW did not result in more precise estimates of the average treatment effect, neither in the simulation nor in the case study.

Conclusion: All chosen approaches suffered in performance due to the small number of events, but the results illustrate that propensity score analyses cannot necessarily improve over other proper causal inference techniques when events are rare.

The authors declare that they have no competing interests.

The authors declare that a positive ethics committee vote has been obtained.