Determination of suitable permutation numbers in comparing independent group means: a monte carlo simulation study

Abstract

In this study, suitable number of permutations were tried to be determined empirically for permutation test, one of whichalternatively used in analysis of variance (ANOVA). Random numbers were generated from (x 3)2 distribution by simulationtechnique, and type I error rates were calculated with their standard errors. For k=2, 3 and 4, data were dealt with what iscommonly faced in practice. Number of observation (n) in each group were taken to be 5, 10 and 15 (except for k = 2). Numberof permutation changes with respect to k and n. For k=2, 3 and 4, required number of permutation was determined to be 15,000,25,000 and 30,000 respectively. In general, there is great advantage of not allowing number of permutation under 10,000, ifpermutation tests are to be used in testing significance of differences of means of groups.