gms | German Medical Science

The idea of using the closed testing procedure for multi-group comparisons goes back to the seventies of the last century [1]. The closed testing procedure tests null hypotheses from a set W that is closed under intersection at the unadjusted level of significance α. A H0 in W is rejected if and only if all H0' in W with H0' ? H0 are rejected at level α. The closed testing procedure controls the familywise alpha error probability by α. If k groups are to be compared with the expected values μ1 ,..., μk, all null hypotheses that can be generated by intersection from the null hypotheses H0: μi= μj , 1≤i<j≤k must be tested. The testing of these null hypotheses is usually done by variance analytical methods (ANOVA). However, it is known that under certain alternative hypotheses the power of ANOVA is lower than that of tests based on the maximum of T-statistics (c.f. [2]). We compare the ANOVA-based closed testing procedure with Tukey's post-hoc test that is based on the maximum range of T-statistics. We show that for a wide range of alternative hypotheses the closed testing procedure is superior to Tukey's post-hoc test.

A major limitation of the closed testing procedure is its computational effort. For example comparisons of k=10 groups require the performance of 115,974 ANOVA tests. We show that by clever programming the computational time for not too large k can be reduced to a tolerable level and thus the closed testing procedure can be used in many situations relevant in practice.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.