gms | German Medical Science

Background and status of (inter)national research: Evidence syntheses provide the basis for evidence-based decision making in healthcare. To judge the certainty of findings in a specific decision context (e.g. jurisdiction), evidence syntheses should consider context suitability. We define context suitability as a generic term to summarize associated concepts and terms like external validity, generalizability, applicability or transferability. There are different criteria, which might be relevant when assessing context suitability.

Question and objective: Our objective was to provide an overview of recommended assessment criteria for assessing context suitability, by reviewing guidance documents from Health Technology Assessment (HTA) and Systematic Review (SR) producing organisations.

Method: We systematically hand searched the webpages of national and international HTA or SR producing organisations for potentially relevant method documents from January to March 2019. Two reviewers independently selected documents according to pre-defined eligibility criteria. One reviewer extracted data in standardized and piloted tables and a second reviewer checked it for accuracy. We extracted all reported assessment criteria literally and assigned them to related domains (e.g. population, setting). We developed the domains inductively. Further, we grouped the extracted assessment criteria by developing a set of domain-specific items (e.g., we grouped “comorbidity” and “co-morbid conditions” as “comorbidities”). The results presented here refer to the assessment of context suitability regarding evidence on effectiveness of health care interventions.

Results: We included 14 method documents from 12 organisations in our synthesis. Most organisations either provide examples, a list of relevant assessment criteria or specific questions to consider when assessing context suitability. We assigned these assessment criteria to the following domains: Population, intervention, comparator, outcome, safety and setting. We identified 57 domain-specific items. Similarities between study setting and target setting regarding population characteristics (e.g., demographics, stage of illness) and intervention characteristics (e.g., treatment regimen, adherence) are the most frequently recommended domains. Further, the healthcare setting (e.g., different standards of care, availability of health technologies) is often mentioned as an important domain. In addition, consideration of the patient-relevance of outcomes is part of some assessment approaches.

Discussion: We present an overview of a number of different assessment criteria recommended by 12 HTA or SR producing organisations for assessing context suitability. The recommended assessment criteria vary between the organisations.

Practical implications: Scientists can use this comprehensive overview as a list of candidate criteria, which could be relevant for assessing context suitability of findings from evidence synthesis to their own research question or decision context.

Appeal for practice in one sentence: From the multitude of potential assessment criteria, scientists should consider those that fit best their research question.