<jupyter_start><jupyter_text>1. Import the necessary packages to read in the data, plot, and create a linear regression model<jupyter_code>import pandas as pd
%matplotlib inline
import matplotlib.pyplot as plt 
import statsmodels.formula.api as smf<jupyter_output>/usr/local/lib/python3.5/site-packages/matplotlib/__init__.py:1035: UserWarning: Duplicate key in file "/Users/zhizhou/.matplotlib/matplotlibrc", line #2
  (fname, cnt))
/usr/local/lib/python3.5/site-packages/matplotlib/__init__.py:1035: UserWarning: Duplicate key in file "/Users/zhizhou/.matplotlib/matplotlibrc", line #3
  (fname, cnt))<jupyter_text>2. Read in the hanford.csv file<jupyter_code>df = pd.read_csv("../data/hanford.csv")
df.head()<jupyter_output><empty_output><jupyter_text>3. Calculate the basic descriptive statistics on the data<jupyter_code>df.describe()<jupyter_output><empty_output><jupyter_text>4. Calculate the coefficient of correlation (r) and generate the scatter plot. Does there seem to be a correlation worthy of investigation?<jupyter_code>df.corr()
df.plot(kind='scatter',x='Exposure',y='Mortality')
print('Yes.')<jupyter_output>Yes.<jupyter_text>5. Create a linear regression model based on the available data to predict the mortality rate given a level of exposure<jupyter_code>lm = smf.ols(formula="Mortality~Exposure",data=df).fit() 
lm.params
intercept, slope = lm.params
df.plot(kind='scatter',x='Exposure',y='Mortality',color='steelblue',linewidth=0)
plt.plot(df["Exposure"],slope*df["Exposure"]+intercept,"-",color="red")<jupyter_output><empty_output><jupyter_text>6. Plot the linear regression line on the scatter plot of values. Calculate the r^2 (coefficient of determination)<jupyter_code>lm.summary()
print("R^2 equals to 0.858.")<jupyter_output>R^2 equals to 0.858.<jupyter_text>7. Predict the mortality rate (Cancer per 100,000 man years) given an index of exposure = 10<jupyter_code>print("The mortality rate of exposure 10 is", 10*slope+intercept)
def get_mr(exposure):
    rate = exposure*slope + intercept
    return rate
get_mr(10)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12367661212704524, 'fraction_numerical': 0.022617901828681425, 'mean_word_length': 7.089494163424124, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '299838', 'license': 'MIT', 'n_tokens_mistral': 756, 'n_tokens_neox': 706, 'n_words': 170, 'path': '06 Logistic Regression/donow/wang_zhizhou_6_donow.ipynb', 'repo_name': 'barjacks/algorithms_mine', 'size': '2078'}