# Linear Regression and Correlation

## Testing the Significance of the Correlation Coefficient (Optional)

The correlation coefficient, r, tells us about the strength and direction of the linear relationship between x and y. However, the reliability of the linear model also depends on how many observed data points are in the sample. We need to look at both the correlation coefficient r and the sample size n, together.
We perform a hypothesis test of the significance of the correlation coefficient to decide whether the linear relationship in the sample data is strong enough to use to model the relationship in the population.
The sample data are used to compute r, the correlation coefficient for the sample. If we had data for the entire population, we could find the population correlation coefficient. But, because we have only sample data, we cannot calculate the population correlation coefficient. The sample correlation coefficient, r, is our estimate of the unknown population correlation coefficient.
1. The symbol for the population correlation coefficient is ρ, the Greek letter rho.
2. ρ = population correlation coefficient (unknown).
3. r = sample correlation coefficient (known; calculated from sample data).
The hypothesis test lets us decide whether the value of the population correlation coefficient ρ is close to zero or significantly different from zero. We decide this based on the sample correlation coefficient r and the sample size n.
If the test concludes the correlation coefficient is significantly different from zero, we say the correlation coefficient is significant.


If the test concludes the correlation coefficient is not significantly different from zero (it is close to zero), we say the correlation coefficient is not significant.



### Performing the Hypothesis Test

1. Null hypothesis: H0: ρ = 0.
2. Alternate hypothesis: H: ρ ≠ 0.
What the Hypothesis Means in Words:
1. Null hypothesis  The population correlation coefficient is not significantly different from zero. There is not a significant linear relationship (correlation) between x and y in the population.
2. Alternate hypothesis  The population correlation coefficient is significantly different from zero. There is a significant linear relationship (correlation) between x and y in the population.
Drawing a Conclusion:There are two methods to make a conclusion. The two methods are equivalent and give the same result.
1. Method 1: Use the 
2. Method 2: Use a table of critical values.
In this chapter, we will always use a significance level of 5 percent, α = 0.05.

### METHOD 1:  Using a p-value to Make a Decision

2. Decision: Reject the null hypothesis.
3. Conclusion: There is sufficient evidence to conclude there is a significant linear relationship between x and y because the correlation coefficient is significantly different from zero.
2. Decision: Do not reject the null hypothesis.
3. Conclusion: There is insufficient evidence to conclude there is a significant linear relationship  between x and y because the correlation coefficient is not significantly different from zero.
You will use technology to calculate the p-value, but it is useful to know that the p-value is calculated using a t distribution with n – 2 degrees of freedom and that the p-value is the combined area in both tails.
An alternative way to calculate the p-value (p) given by LinRegTTest is the command 2*tcdf(abs(t),10^99, n–2) in 2nd DISTR.
2. Consider the third exam/final exam example.
3. The line of best fit is ŷ = 173.51 + 4.83x, with r = 0.6631, and there are n = 11 data points.
4. Can the regression line be used for prediction? Given a third exam score (x value), can we
use the line to predict the final exam score (predicted y value)?
H0: ρ = 0
H: ρ ≠ 0
α = 0.05
1. The p-value is 0.026 (from LinRegTTest on a calculator or from computer software).
2. The p-value, 0.026, is less than the significance level of α = 0.05.
3. Decision: Reject the null hypothesis H0.
4. Conclusion: There is sufficient evidence to conclude there is a significant linear relationship between the third exam score (x) and the final exam score (y) because the correlation coefficient is significantly different from zero.
Because r is significant and the scatter plot shows a linear trend, the regression line can be used to predict final exam scores.

### METHOD 2: Using a Table of Critical Values to Make a Decision

The 95 Percent Critical Values of the Sample Correlation Coefficient Table () can be used to give you a good idea of whether the computed value of r is significant. Use it to find the critical values using the degrees of freedom, df = n – 2. The table has already been calculated with α = 0.05. The table tells you the positive critical value, but you should also make that number negative to have two critical values. If r is not between the positive and negative critical values, then the correlation coefficient is significant. If r is significant, then you may use the line for prediction. If r is not significant (between the critical values), you should not use the line to make predictions.

### Third Exam vs. Final Exam Example: Critical Value Method

Consider the third exam/final exam example.
The line of best fit is: ŷ = –173.51 + 4.83x, with r = .6631, and there are n = 11 data points. Can the regression line be used for prediction? Given a third exam score (x value), can we use the line to predict the final exam score (predicted y value)?

1. H0: ρ = 0
2.  H: ρ ≠ 0 
3. α = 0.05
1. Use the 95 Percent Critical Values table for r with df = n – 2 = 11 – 2 = 9.
2. Using the table with df = 9, we find that the critical value listed is 0.602. Therefore, the critical values are ±0.602.
3. Since 0.6631 > 0.602, r is significant.
4. Decision: Reject the null hypothesis.
5. Conclusion: There is sufficient evidence to conclude there is a significant  linear relationship between the third exam score (x) and the final exam score (y) because the correlation coefficient is significantly different from zero.
Because r is significant and the scatter plot shows a linear trend, the regression line can be used to predict final exam scores.

### Assumptions in Testing the Significance of the Correlation Coefficient

Testing the significance of the correlation coefficient requires that certain assumptions about the data be satisfied. The premise of this test is that the data are a sample of observed points taken from a larger population.  We have not examined the entire population because it is not possible or feasible to do so.  We are examining the sample to draw a conclusion about whether the linear relationship that we see between x and y in the sample data provides strong enough evidence that we can conclude there is a linear relationship between x and y in the population.
The regression line equation that we calculate from the sample data gives the best-fit line for our particular sample. We want to use this best-fit line for the sample as an estimate of the best-fit line for the population. Examining the scatter plot and testing the significance of the correlation coefficient helps us determine whether it is appropriate to do this.
2. There is a linear relationship in the population that models the sample data. Our regression line from the sample is our best estimate of this line in the population.
3.  The y values for any particular x value are normally distributed about the line. This implies there are more y values scattered closer to the line than are scattered farther away. Assumption 1 implies that these normal distributions are centered on the line; the means of these normal distributions of y values lie on the line.
4. Normal distributions of all the y values have the same shape and spread about the line.
5. The residual errors are mutually independent (no pattern).
6. The data are produced from a well-designed, random sample or randomized experiment.

### Chapter Review

Linear regression is a procedure for fitting a straight line of the form ŷ = a + bx to data. The conditions for regression are as follows:
1. Linear: In the population, there is a linear relationship that models the average value of y for different values of x.
2. Independent: The residuals are assumed to be independent.
3. Normal: The y values are distributed normally for any value of x.
4. Equal variance: The standard deviation of the y values is equal for each x value.
5. Random: The data are produced from a well-designed random sample or a randomized experiment.
The slope b and intercept a of the least-squares line estimate the slope β and intercept α of the population (true) regression line. To estimate the population standard deviation of y (σ) use the standard deviation of the residuals: 
 . The variable ρ (rho) is the population correlation coefficient. To test the null hypothesis, H0: ρ = hypothesized value, use a linear regression t-test.  The most common null hypothesis is H0: ρ = 0, which indicates there is no linear relationship between x and y in the population. The TI-83, 83+, 84, 84+ calculator function LinRegTTest can perform this test (STATS, TESTS, LinRegTTest).

### Formula Review

Least-Squares Line or Line of Best Fit: 

 
where a is the y-intercept and b is the slope.
Standard Deviation of the Residuals: 

 
where SSE = sum of squared errors, and
n = the number of data points.