gms | German Medical Science

Background/research question: Evidence syntheses provide the basis for evidence-based decision making in healthcare. To judge the certainty of findings in a specific decision context, evidence syntheses should consider context suitability (i.e. generalisability, external validity, applicability or transferability). Our objective was to determine the status quo and to provide an overview of existing methodological recommendations on this topic, by reviewing guidance documents from Health Technology Assessment (HTA) and Systematic Review (SR) producing organisations.

Methods: We systematically hand searched the webpages of national and international evidence synthesis producing organisations for potentially relevant methods documents from January to March 2019. Two reviewers independently selected documents according to pre-defined eligibility criteria (e.g. documents include recommendations for appraising context suitability in effectiveness assessments). One reviewer extracted data in standardised and piloted tables and a second reviewer checked it for accuracy. We extracted and aggregated data according to the following issues: the target audience of the evidence synthesis product, the types of considered interventions, the underlying terminology (e.g. applicability or generalisability), information on assessment approach concepts (e.g. assessment level, assessment standardisation) and the relevant assessment criteria. We performed a thematic analysis to identify and summarise the main themes regarding recommended context suitability assessments. Additionally, we analysed similarities and differences between the recommendations. The results presented here refer to the assessment of context suitability regarding evidence on effectiveness of health care interventions.

Results: We included 14 methods documents from twelve organisations in our synthesis. The included methodological recommendations vary widely regarding the underlying terminology, the assessment approach concepts (e.g. assessment standardisation) and the content of assessments (e.g. assessment criteria).

Conclusion: It appears justified that the assessment of context suitability is somewhat heterogeneous because of the need to tailor the assessment to different decision contexts and the varying relevance of aspects in relation to the medical area. Many differences between the recommendations seem unexplained. More harmonization is desirable and appears possible.

Competing interests: The authors declare that they have no conflicts of interests.