gms | German Medical Science

Background: There is a consensus that an explanatory trial should confirm either superiority or equivalence or non-inferiority when comparing the effects of two or more interventions. It has to be completed under ideal study conditions and it is expected to guarantee high internal validity. A consensus on the optimal design of a pragmatic trial is still missing. Researchers agree that a pragmatic trial should assess effectiveness but don’t agree on the precise definition of effectiveness, the need of randomization, the adequate comparator, and other necessary conditions including the appropriate evaluation.

Objective: To propose a concept which demonstrates different goals and different results of efficacy and effectiveness trials. Depending on the differences of expected functions adequate tools should be selected to differentiate between efficacy and effectiveness.

Method: First, the expected differences of efficacy and effectiveness and the most frequent forms of bias in clinical trials were identified from the published literature. Second, based on these data the goals of explanatory and of pragmatic trials were defined. Third, the information was identified which is necessary to meet the goals of explanatory or pragmatic trials but also to avoid the most frequent forms of bias.

Results: Two strategies, to complete an experimental and a pragmatic trial are suggested. Both strategies are described in 13 steps, ask different questions, use different methods, and provide different answers which can be used for different decisions. Five of 13 steps are identical in both strategies: 1) the definition of the primary and secondary study questions, 2) the distinction of primary and secondary outcomes, 3) the definition of inclusion criteria, 4) the follow up long enough to detect adverse effects and outcomes, 5) the use of statistical methods to reduce effects of confounders but also the limitation of statistical tests to results which are clinically relevant (saving statistical energy). The eight remaining of the 13 steps which are different in explanatory and pragmatic trials are: 1) the definition of exclusion criteria, 2) the precise pre-study definition of the compared treatment options, 3) the selection of the study design depending on the study question, 4) the definition of the expected difference in the experimental and control arm, 5) the definition of the alpha-, beta-error and power calculation, 6) the purpose of the informed consent, 7) the allocation of patients to the treatment options, 8) the need of stratification, and finally 8) the need of an intent-to-treat analysis.

Discussion: The application of the two proposed strategies will be feasible as both proposals are strictly oriented on either the necessary conditions for completion of a Randomized Controlled Trial or the conditions for provision of standard health care according to the principles of Good Clinical Practice. According to several authors the purpose of explanatory and pragmatic trials is different. The aim of an explanatory trial is the basic demonstration of efficacy i.e. it has to be demonstrated that the new principle works under ideal study conditions. The ideal study condition has to be free from possible confounders. It is recommended to select the optimal model to demonstrate efficacy under ideal study conditions as the aim of this experiment is just to demonstrate that the new principle can theoretically do what it is expected to do. Consequently, it will be enough to demonstrate efficacy in a single explanatory study. A second study may be requested just to reduce the risk of a random result. Subsequent studies should be pragmatic trials i.e. descriptive trials. Pragmatic trials i.e. the systematic description of outcomes under real world conditions should be applied to confirm that the effects which are expected from the results of efficacy studies can indeed be observed under real world conditions. Real world conditions means that confounders cannot be excluded. We have to challenge the unconsciously presumed hypothesis “efficacy can predict effectiveness”.

Practical implications: The proposal will contributes to close the efficacy-effectiveness gap will have consequences on health care research and health care provision. Strong effects have to be expected on economic but not on medical outcomes of care. The proposal can definitely increase the efficiency of care as the standardized assessment of effectiveness will quickly identify efficient and inefficient methods in prevention, cure, and care. As the health care market is by far the strongest segment of our national economies the necessary political decisions have to be considered before the new strategy can be implemented. Welfare of a population will largely depend on both a successful health care market and an efficient health care system.