Generalized p-value

In statistics, a generalized "p"-value is an extended version of the classical "p"-value, which except in a limited number of applications, provides only approximate solutions.
Conventional statistical methods do not provide exact solutions to many statistical problems, such as those arising in mixed models and MANOVA, especially when the problem involves a number of nuisance parameters. As a result, practitioners often resort to approximate statistical methods or asymptotic statistical methods that are valid only when the sample size is large. With small samples, such methods often have poor performance. Use of approximate and asymptotic methods may lead to misleading conclusions or may fail to detect truly significant results from experiments.
Tests based on generalized "p"-values are exact statistical methods in that they are based on exact probability statements. While conventional statistical methods do not provide exact solutions to such problems as testing variance components or ANOVA under unequal variances, exact tests for such problems can be obtained based on generalized "p"-values.
In order to overcome the shortcomings of the classical "p"-values, Tsui and Weerahandi extended the classical definition so that one can obtain exact solutions for such problems as the Behrens–Fisher problem and testing variance components. This is accomplished by allowing test variables to depend on observable random vectors as well as their observed values, as in the Bayesian treatment of the problem, but without having to treat constant parameters as random variables.
Example.
To describe the idea of generalized "p"-values in a simple example, consider a situation of sampling from a normal population with the mean formula_1, and the variance formula_2. Let formula_3 and formula_4 be the sample mean and the sample variance. Inferences on all unknown parameters can be based on the distributional results
and
Now suppose we need to test the coefficient of variation, formula_7. While the problem is not trivial with conventional "p"-values, the task can be easily accomplished based on the generalized test variable
where formula_9 is the observed value of formula_3 and formula_11 is the observed value of formula_12. Note that the distribution of formula_13 and its observed value are both free of nuisance parameters. Therefore, a test of a hypothesis with a one-sided alternative such as formula_14 can be based on the generalized "p"-value formula_15, a quantity that can be easily evaluated via Monte Carlo simulation or using the non-central t-distribution.