gms | German Medical Science

Background/research question: Randomised controlled trials (RCTs) are considered to estimate intervention effects most reliably, yet design flaws can lead to inaccuracies [1]. In nutrition RCTs, the challenges of achieving blinding due to the nature of dietary interventions, along with many trials remain unregistered, may increase the risk of bias (RoB) [2]. Assessing RoB with adequate tools is crucial to identify study limitations. This meta-epidemiological study aims to evaluate the average effect difference due to bias associated with methodological characteristics in nutrition RCTs by using the RoB 2 tool [3].

Methods: Individual RCTs published between 1969 and 2018 were identified from a representative sample of nutrition systematic reviews. Two authors independently conducted extensive data extraction and RoB assessments. RoB domains evaluated included biases arising from the randomisation process, deviations from the intended interventions, missing outcome data, outcome measurement, and selective reporting. Average estimates due to bias (ratio of risk ratio [RRR]) were computed through meta-analyses using a random-effects model, comparing RCTs rated as ‘high risk’ or ‘some concerns’ to those rated ‘low risk’ of bias. Subgroup analyses were conducted to investigate differences across individual RoB domains, types of interventions and outcomes.

Results: We included 120 outcome-specific RoB assessments. Among these, 84 assessments (70.0%) were rated as having ‘some concerns’, 22 (18.3%) as ‘low risk’, and 14 (11.7%) as ‘high risk’ of bias. Overall RoB ratings did not affect intervention effect estimates (RRR 0.99, 95% confidence interval [CI] 0.85 to 1.14; I²=36%; heterogeneity estimator [τ²]=0.03; prediction interval [PI] 0.66 to 1.47). Most RoB domains did not reveal differences in effect estimates, except for trials with biases related to deviations from the intended intervention (RRR 1.29, 95% CI 1.13 to 1.48; I2=2%; τ²=0.01; PI 0.97 to 1.72). These findings were confirmed in subgroup and sensitivity analyses.

Conclusion: Several methodological characteristics in nutrition RCTs assessed with the RoB 2 tool may not substantially over- or underestimate intervention estimates. However, biases arising from deviations from intended interventions may lead to exaggeration. Replication of our findings is necessary, particularly given the unexpected results of potential underestimation of trials with bias from deviations from intended interventions.

Competing interests: The authors have no relevant financial or non-financial interest to disclose.