Some Implications of an Alternative Definition of the Multiple Comparison Problem

An approach to multiple comparisons defines the error control objective as assuring with given probability that k or fewer out of N specified linear combinations of p means are significant when an overall null hypothesis is true, or that k or fewer confidence interval statements are incorrect. The case k = 0 is basically the Scheffé (1953) method except for the consideration of a specified set of N linear combinations rather than all possible linear combinations. A theoretical result is obtained under normal theory and we assume an infinite number of linear combinations with normalized coefficients, that is the sum of squares is unity, uniformly distributed over the unit p-sphere. For a finite number N of specified linear combinations the problem is reformulated in a way which in principle would allow computation of the exact distributions required to provide critical values. In general, the computations required are so complex that they are impractical, but a Monte Carlo approach is straightforward.