gms | German Medical Science

Background: Randomized controlled trials (RCTs) are the only way to control for unmeasured confounding and consequently to ensuring internal validity. However RCTs are usually expensive. The main reason is the implementation of an infrastructure for data acquisition, the process of data acquisition and patient recruitment. Moreover the lack of external validity is often criticized.

A registry is defined as „a file of documents containing uniform information about individual persons, collected in a systematic and comprehensive way, in order to serve a predetermined purpose” (AHRQ). Once are implemented registries enable rapid recruitment of patients and cost-effective acquisition of “real world” data. In theory, the pros of registries should correspond almost exactly to the drawbacks of RCTs. This led to the attention to registry RCT (rRCTs).

Objective: The objective was to get insights into methodological aspects and feasibility challenges of rRCTs. Furthermore we wanted to analyse for which research questions rRCTs were in particular appropriate.

Methods: A systematic literature search was performed in MEDLINE (via Pubmed). Furthermore the references of discussion papers about rRCTs were cross-checked. We included RCTs or cluster RCTs which gathered at least one outcome using a registry. Compliance of studies with inclusion criteria were assessed independently by two reviewers.

Information on patients, health technology (HT), outcomes, study design, methodological aspects, the registry and country/setting were extracted in standardized tables by one reviewer and checked by a second reviewer.

Results: We identified 62 relevant rRCTs. The majority of trials evaluated HTs for prevention. Screening and vaccination were evaluated most often. Apart from prevention medical devices, surgical procedures, and coordination and organization of care were frequently evaluated. Patient recruitment was often based on registries. Sample size of most rRCTs was very large with a great proportion of mega-trials. Different types of registries were used; mostly national patient registries, vaccination registries, death registries and patient registries on chronic diseases. Predominately the registry was not the only source of data but was used to gather long term outcomes. The outcome that was most often gathered in the registry was mortality. In almost all publications sufficient information on completeness and quality of data could not be found. Most studies were performed in Scandinavia followed by USA and Canada. In studies that did not use the registry as the only data source, records were mostly linked via the personal or social insurance number. In studies that exclusively used registry data mostly a randomization indicator was included in the registry itself.

Discussion: A considerable amount of trials exist that use registries as data sources. rRCTs are often large and seem especially used to gather long-term outcomes. This might be explained by the less expense for data acquisition. Furthermore in many of the evaluated HT (e.g. prevention) event rates are low and thus large sample sizes are needed to prove statistical confidence.

For trials that are used for the approval of drugs high requirements for data acquisition exist. Normally this requirements are weaker or even do not exist for other HTs (e.g. medical devices). This fact might explain the great amount of trials on non-pharmacological HTs.

The doubts on data quality seem reasonable. Also our analysis showed that completeness and quality of data is almost never assessable because it is neither reported in the publication nor can be easily assessed in other ways.

The overrepresentation of certain countries indicates that in countries with restrictive data privacy and without national registries record linkage might be a barrier to perform rRCTs.

Implications for practice: rRCT seem a valuable tool to gather effectiveness data. Our analyses showed that rRCTs allow an efficient recruitment of large sample sizes and a long-term assessment of patient important outcomes. The external validity of results might be further increased if combined with the methodological approaches of pragmatic trials. For sufficient HT assessment information on effectiveness and efficacy is needed. rRCTs can therefore also contribute to an increase of the external validity of HT assessment.

A premise for acceptance of registry data (e.g. for drug approval) is an increase in transparent documentation of completeness and quality of data.