gms | German Medical Science

The conduct of replication studies not only plays an important role in assessing the credibility of scientific discoveries, but is often a regulatory requirement in the case of clinical research. Although most researchers agree on the importance of replication studies, there is currently no agreement on a statistical criterion for replication success. A reverse-Bayes method called the sceptical p-value [1] has been proposed for this purpose; the inversion of Bayes' theorem allows to mathematically formalize the notion of scepticism, which in turn can be used to assess the agreement between the findings of an original study and its replication. However, despite its Bayesian nature, the method relies on tail-probabilities and confidence intervals as primary inference tools. Here we present an extension that uses Bayes factors instead as a means to quantify evidence. This leads to a new measure for evaluating replication success, the sceptical Bayes factor. Conceptually, the sceptical Bayes factor reveals the level of evidence at which the replication result can convince a sceptic who did not trust the original finding. While the sceptical p-value can only quantify the conflict between the sceptic and the observed replication data, the sceptical Bayes factor also takes into account how likely the data are if one would trust the original finding. A case study from the Reproducibility Project: Cancer Biology shows the advantages of the proposed method for the quantitative assessment of replicability.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.