gms | German Medical Science

Background: In multiple testing problems with composite null hypotheses, we are interested in estimating the proportion of the true nullhypotheses with the Schweder-Spjøtvoll estimator. The latter utilizes marginal p-values and only works properly if the p-values that correspond to the true null hypotheses are uniformly distributed on [0,1] (Uni[0,1]). We introduce randomized p-values that are closer than conventional p-values to Uni[0,1], and give an example in a replicability analysis model.

Methods: In case of composite null hypotheses, marginal p-values are usually computed under least favourable parameter configurations (LFCs) resulting in p-values that are larger than Uni[0,1] under non-LFCs in the null hypotheses. We provide a range of randomized p-value sets, that include the LFC-based p-values, and show that there exists a set of p-values that, if utilized, minimizes the bias of the Schweder-Spjøtvoll estimator.

Results: These bias-optimized randomized p-values differ from the LFC-based p-values, if the latter are much larger than Uni[0,1] under null hypotheses. In general, apart from the bias, the use of our randomized p-values also reduces the mean squared error if utilized in the Schweder-Spjøtvoll estimator.

Conclusion: More accurate estimations of the proportion of true null hypotheses are not only valuable in themselves, but can also improve the power of existing mulitple testing procedures. We provide a way of decreasing the bias of an established estimator by providing marginal p-values that have more desirable properties under null hypotheses.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.