gms | German Medical Science

Low statistical power in preclinical experiments has been repeatedly pointed out as a roadblock to successful replication and translation. If only a small number of tested interventions is effective (i.e. pre-study odds are low), researchers should increase the power of their experiments to detect those true effects. This, however, contradicts ethical and budget constraints. To increase reproducibility and scientific value of preclinical experiments under these constraints, it is necessary to devise strategies that improve the efficiency of confirmatory studies.

To this end, we explore different approaches to preclinical animal experiments via simulations. We model two preclinical research trajectories from the exploratory stage to the results of a within-lab replication study based on empirical pre-study odds. Along the trajectory, there are different ways to increase the probability of not missing potentially meaningful effects. In a first step, a decision is made based on exploratory data whether to continue to a replication. One trajectory (T1) employs the conventional significance threshold for this decision. The second trajectory (T2) uses a more lenient threshold based on an a priori determined smallest effect size of interest (SESOI). The sample size of a potential replication study is calculated via a standard power analysis using the initial exploratory effect size (T1) or using a SESOI (T2). The two trajectories are compared regarding the number of experiments proceeding to replication, number of animals tested in the replication, and positive predictive value (PPV) across the trajectory.

Our simulations show that under the conventional significance threshold, only 32 percent of the initial exploratory experiments progress to the replication stage. Using the decision criterion based on a SESOI, 65 percent of initial studies proceed to replication. T1 results in the lowest number of animals needed for replication (n = 7 per group) but yields a PPV that is below pre-study odds. T2 increases PPV above pre-study odds while keeping sample size at a reasonably low number (n = 23 per group).

Our results reveal that current practice, represented by T1, results in a large number of false negatives and impedes efforts to replicate preclinical experiments. Optimizing decision criteria and experimental design by employing easily applicable variations as shown in T2 increases researchers' chances to detect true effects. Importantly, T2 keeps tested animal numbers low while generating more robust preclinical evidence that may ultimately benefit translation.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.