<jupyter_start><jupyter_text>STAT 5310 – Regression Analysis Lab Exercise/Homework 2  1. Develop a regression model to predict the average number of hits (Y) from the average salary for the 10 highest paid hitters from each MLB team (X) based on data from the 2013 season.The data is in the file Hitters<jupyter_code>hitters <- fread('https://raw.githubusercontent.com/wilsonify/AppliedRegression/master/data/Hitters.csv')
names(hitters) <- names(hitters) %>% gsub('[ |(|)|$]','',.)
hitters %>% head()<jupyter_output><empty_output><jupyter_text>a) Fit a straight line regression model between Y and X.  Decide whether $\beta_1$, the coefficient of X, is statistically significantly different from zero or not.  Give a reason to support your answer.<jupyter_code>fit <- lm(hitters$AverageNumberofHits ~ hitters$AverageSalary2013M)
summary_of_fit <- summary(fit)
summary_of_fit$coefficients["hitters$AverageSalary2013M","Pr(>|t|)"]<jupyter_output><empty_output><jupyter_text>$\beta_1$ is not significatnly different from zero.Under the null hypothesis that $\beta_1 = 0$, we would observe a more extreme $\beta_1$ 13% of the time.<jupyter_code>plot( y = hitters$AverageNumberofHits
     ,x = hitters$AverageSalary2013M
    )<jupyter_output><empty_output><jupyter_text>b) Examine the studentized residuals for the fitted model in part a.  Identify any points which are outliers (that is, have studentized residuals values more extreme than 3.<jupyter_code>studentized_residuals <- studres(fit)
qqnorm(studentized_residuals)
qqline(studentized_residuals)<jupyter_output><empty_output><jupyter_text>There is exactly one outlier with studentized residual of -4.36  2. In order to investigate the feasibility of starting a Sunday edition for a large metropolitan newspaper, information was obtained from a sample of 34 newspapers concerning their daily and Sunday circulations (in thousands) The data is in the file Newspaper. *(Source:  Gale Directory of Publications, 1994).*<jupyter_code>newspaper <- fread('https://raw.githubusercontent.com/wilsonify/AppliedRegression/master/data/Newspaper.csv')
newspaper %>% head()<jupyter_output><empty_output><jupyter_text>a)Construct a scatter plot of Sunday circulation versus daily circulation.Does the plot suggest a linear relationship between daily and Sunday circulation?  Do you think this is a plausible relationship?<jupyter_code>plot(y=newspaper$Sunday,x=newspaper$Daily)<jupyter_output><empty_output><jupyter_text>The plot of sunday circualtion vs daily circulation does suggest a linear relationship.It seems very plausible that, in reality, newspapers with a large circulation on weekdays would also have a large circulation on sunday and vis-versa for newspapers with a small circulation.  b)Fit a regression line predicting Sunday circulation from daily circulation.<jupyter_code>fit <- lm(data =newspaper, Sunday~Daily)
plot(y=newspaper$Sunday,x=newspaper$Daily)
abline(fit)<jupyter_output><empty_output><jupyter_text>c)Obtain the 95% confidence intervals for β0 and β1.<jupyter_code>summary_of_fit <- summary(fit)
summary_of_fit$coefficients
alpha = 1.0 - 0.95
half_alpha = alpha / 2.0
z = qnorm(1-half_alpha)


beta0_center <- summary_of_fit$coefficients["(Intercept)","Estimate"]
beta0_SE <- summary_of_fit$coefficients["(Intercept)","Std. Error"]
beta0_error_bound <- z * beta0_SE
beta0_CI <- c(beta0_center-beta0_error_bound,beta0_center+beta0_error_bound) %>% round(2)
print("95% confidence interval for the intercept, B0 is ")
beta0_CI

beta1_center <- summary_of_fit$coefficients["Daily","Estimate"]
beta1_SE <- summary_of_fit$coefficients["Daily","Std. Error"]
beta1_error_bound <- z * beta1_SE
beta1_CI <- c(beta1_center-beta1_error_bound,beta1_center+beta1_error_bound) %>% round(2)
print("95% confidence interval for the slope, B1 is ")
beta1_CI<jupyter_output><empty_output><jupyter_text>d) Is there a significant relationship between Sunday circulation and daily circulation?Justify your answer by a statistical test.Indicate what hypothesis you are testing and your conclusion. There a significant relationship between Sunday circulation and daily circulation.If there were no relationship (H0: $\beta_1 = 0$), we would expect a more extreme coefficient only 0.00000000000000006% of the time.  e)What proportion of the variability in Sunday circulation is accounted for by daily circulation?<jupyter_code>summary_of_fit$adj.r.squared<jupyter_output><empty_output><jupyter_text>91.5% of the variablitity in sunday ciculation can be explained by a linear relashionship between daily circulation and sunday circulation.  f)Provide an estimate (based on a 95% confidence level) for the average Sunday circulation of newspapers with daily circulation of 500,000.<jupyter_code>alpha = 1.0 - 0.95
half_alpha = alpha / 2.0
z = qnorm(1-half_alpha)
newdata <- data.frame(Daily=c(500000))
center <- predict(fit, newdata = newdata)
sigma <- summary_of_fit$sigma
error_bound <- z * sigma
CI <- c(center-error_bound,center+error_bound)
CI<jupyter_output><empty_output><jupyter_text>At a 95% confidence level, the average Sunday circulation of all newspapers with daily circulation of 500,000 will be between 669700 and 670100.  g)The particular newspaper that is considering a Sunday edition has a daily circulation of 500,000.  Provide an interval estimate (based on a 95% confidence level) for the predicted Sunday circulation of this paper.How does this interval differ from that given in part f?<jupyter_code>predict(fit, newdata = data.frame(Daily=c(500000)), interval="predict",level = 0.95)<jupyter_output><empty_output><jupyter_text>At a 95% confidence level, the Sunday circulation of a particular newspaper with daily circulation of 500,000 will be between 598000 and 742000.Generally, there is more (tighter) confidence in predicting the average of all newspapers than in predicting the circulation of a particular newspaper.  h)Another newspaper considering a Sunday edition has a daily circulation of 2,000,000.Provide an interval estimate for the predicted Sunday circulation for this paper?How does this interval compare with the one given in part g?  Do you think it is likely to be accurate?<jupyter_code>predict(fit, newdata = data.frame(Daily=c(2000000)), interval="predict",level = 0.95)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08394276629570747, 'fraction_numerical': 0.032114467408585055, 'mean_word_length': 6.643985419198056, 'pattern_counts': {'":': 0, '<': 70, '<?xml version=': 0, '>': 55, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '573895', 'license': 'MIT', 'n_tokens_mistral': 2037, 'n_tokens_neox': 1808, 'n_words': 639, 'path': 'hw/Grp6.HW2.wilson.ipynb', 'repo_name': 'wilsonify/AppliedRegression', 'size': '6290'}