gms | German Medical Science

Background/research question: As part of the update of the German/Austrian list on potentially inadequate medications for elderly patients (PRISCUS) we performed systematic reviews (SRs) on adverse events in the elderly for various medications. Evidence on harms frequently originates from non-randomized studies (NRS) and is often highly statistically imprecise. In addition, harms are often subgroup specific (e.g. elderly population) and thus information from subgroup analysis must be considered. When rating the certainty of evidence (CoE) applying the GRADE standard methodological challenges occur. First, CoE might tend to be systematically lower for harms compared to benefits. Second, differences in CoE can be diluted because of floor effects, particularly “very low” CoE ratings.We introduce a GRADE adaption for situations, where only harms are assessed with the aim to tackle the abovementioned challenges.

Methods: We systematically assessed each of the GRADE criteria for common problems arising when assessing harms. Criteria were adapted if the adaption could be supported by epidemiological reasoning. We piloted the adaptions using a convenience sample of 3 SRs and subsequently proofed our concept by applying the approach to 15 SRs.

Results: The main adaptions were (reasoning):

• Starting point: evidence from NRS does not usually start low (usually harms do not predict the received intervention essentially)
• Imprecision: stronger imprecision was excepted for severe, rare harms, if the observed effect could be justified by casual explanation (it can be deemed improbable that the effect is really in the opposite direction although confidence intervals suggest this)
• Publication bias: in case of publication bias CoE was rated up (not publishing studies showing large effects for harms against the intervention would result in underestimation of the effect)
• Large effect: the cutoff for a large effect was decreased for NRS with adequate confounding adjustment (methodological studies that have determined the threshold for large effects were based on unadjusted estimates)

In addition, we rated up the CoE if the p-value of a test for interaction from a well-designed subgroup analysis was <0.05.

We found no inconsistencies or counterintuitive overestimations of the CoE.

Conclusion: Conclusion

Our study suggests that there is potential for developing adaptions of the GRADE criteria, which can overcome the challenges of assessing harms. Further research is necessary to develop consensus recommendations for rating the CoE on harms.

Competing interests: BMBF Förderkennzeichen 01KX1812