therapies, therapy A and therapy B, are being compared. (Keep in mind the sampling of beads just discussed.) Before beginning our study, we can list the four possible study outcomes: It is possible that in reality there is no difference in efficacy between therapy A and therapy B (i.e., therapy A is no better and no worse than therapy B), and when we do our study we correctly conclude on the basis of our samples that the two groups do not differ. It is possible that in reality there is no difference in efficacy between therapy A and therapy B (i.e., therapy A is no better and no worse than therapy B), but in our study we found a difference between the groups and therefore concluded, on the basis of our samples, that there is a difference between the therapies. This conclusion, based on our samples, is in error. It is possible that in reality there is a difference between therapy A and therapy B, but when we examine the groups in our study we find no difference between them. We therefore conclude, on the basis of our samples, that there is no difference between therapy A and therapy B. This conclusion is in error. It is possible that in reality there is a difference between therapy A and therapy B, and when we examine the groups in our study we find that they differ. On the basis of these samples, we correctly conclude that therapy A differs from therapy B.
These four possibilities constitute the universe of outcomes after we complete our study. Let us look at these four possibilities as presented in a 2 × 2 table: Two columns represent reality-either therapy A differs from therapy B or therapy A does not differ from therapy B. The two rows represent our decision: We conclude either that they differ or that they do not differ. In this figure, the four possibilities that were just listed are represented as four cells in the 2 × 2 table. If there is no difference, and on the basis of the samples included in our study we conclude there is no difference, this is a correct decision (cell a). If there is a difference, and on the basis of our study we conclude that there is a difference (cell d), this too is a correct decision. In the best of all worlds, all of the possibilities would fall into one of these two cells. Unfortunately, this is rarely, if ever, the case. There are times when there is no difference between the therapies, but on the basis of the samples of subjects included in our study, we erroneously conclude that they differ (cell c). This is called a type I error. It is also possible that there really is a difference between the therapies, but on the basis of the samples included in our study we erroneously conclude that there is no difference (cell b); this is called a type II error. (In this situation, the therapies differ, but we fail to detect the difference in our study samples.) The probability that we will make a type I error is designated α, and the probability that we will make a type II error is designated β.
α is the so-called P value, which is seen in many published papers and has been sanctified by many years of use. When you see " P < .05," the reference is to α. What does P < .05 mean? It tells us that we have concluded that therapy A differs from therapy B on the basis of the sample of subjects included in our study, which we found to differ. The probability that such a difference could have arisen by chance alone, and that this difference between our groups does not reflect any true difference between therapies A and B, is only .05 (or 1 in 20).
How do these concepts help us to arrive at an estimate of the sample size that we need? If we ask the question, "How many people do we have to study in a clinical trial?" we must be able to specify a number of items
Use the same method as described in Figure 7-6 . Use the standard normal distribution (Table A-2).
Hypothesis testingHypothesis testingDraw inferences about a populationbased on a sampleTesting a claim about a property ofa population
Statistical Inference∗ Inferences about a population are made onthe basis of results obtained from a sampledrawn