gms | German Medical Science

Background: In a recent systematic review of cluster-randomized trials comparing a complex intervention to routine care in a general practice setting, we found that only 14% of studies were able to show the effectiveness of the intervention for a primary patient-relevant outcome. Given that complex interventions are time-consuming and expensive to both plan and conduct, we sought to provide a possible explanation for this phenomenon by investigating whether the assumptions made during the planning stage of these studies are unrealistic.

Material/Methods: Trials included in the systematic review that reported sample size calculations for a patient-relevant primary outcome and accounted for the clustered nature of the data therein were analyzed further. We extracted detailed information on the sample size calculations regarding intra-cluster correlation and treatment effect and, where available, compared the assumptions made in the sample size calculations with the obtained results by means of scatter plots and relative differences.

Results: Eighteen of the initial 29 studies met our tightened inclusion criteria and data from sample size calculations for 21 outcomes was available for analysis. Comparison of assumptions and obtained results with respect to intra-cluster correlation and treatment effect was possible for 13 outcomes from 11 studies and all 21 outcomes from 18 studies, respectively. The intra-cluster correlation was assumed conservatively in most studies, whereas only one study observed the expected treatment effect. For some outcomes, assumptions about baseline values for continuous outcomes were also provided; these differed from the observed results by up to 87%, even though baseline differences between the groups were small.

Conclusion: Whereas the extent of clustering was realistically anticipated in most studies, investigators seem overly optimistic with their expectation of the interventions.