<jupyter_start><jupyter_text>Linear Regression (single variable) demoThe dataset used is the Leinhardt Infant Mortality dataset from the R CAR package. Here we get the same data from the [Rdatasets repository](https://vincentarelbundock.github.io/Rdatasets/).The [Documentation for Leinhardt dataset](https://vincentarelbundock.github.io/Rdatasets/doc/carData/Leinhardt.html) can also be found in this repository. Problem DefinitionWe want to predict the infant mortality response variable by using one or more explanatory variables available in the Leinhardt dataset.<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import statsmodels.api as sm
%matplotlib inline
import warnings
warnings.filterwarnings("ignore")<jupyter_output><empty_output><jupyter_text>Load and explore data<jupyter_code>try:
    leinhardt_df = pd.read_csv("Leinhardt.csv")
except:
    leinhardt_df = pd.read_csv("https://vincentarelbundock.github.io/Rdatasets/csv/carData/Leinhardt.csv")
    leinhardt_df.to_csv("Leinhardt.csv")
leinhardt_df.head()<jupyter_output><empty_output><jupyter_text>Investigate correlation between pairs of variables<jupyter_code>leinhardt_df["region"].unique()
leinhardt_df["oil"].unique()
leinhardt_df.loc[leinhardt_df["region"] == "Asia", "region_i"] = 0
leinhardt_df.loc[leinhardt_df["region"] == "Europe", "region_i"] = 1
leinhardt_df.loc[leinhardt_df["region"] == "Americas" "region_i"] = 2
leinhardt_df.loc[leinhardt_df["region"] == "Africa", "region_i"] = 3

leinhardt_df.loc[leinhardt_df["oil"] == 'no', 'oil_i'] = 0
leinhardt_df.loc[leinhardt_df["oil"] == 'yes', 'oil_i'] = 1

_ = pd.plotting.scatter_matrix(leinhardt_df, diagonal="hist", figsize=(10,10))<jupyter_output><empty_output><jupyter_text>Investigate correlation between Infant Mortality and IncomeFrom the scatter plots, there is negative correlation, but correlation doesn't appear to be linear. Thus Linear Regression on these variables is not appropriate.However, both distributions are right skewed, which is a strong hint that they might become linearly related on a log scale. On adding these log scaled columns, we see a strong linear relationship.<jupyter_code>plt.scatter(leinhardt_df["income"], leinhardt_df["infant"])
plt.xlabel("income")
plt.ylabel("infant")
_ = plt.show()
plt.hist(leinhardt_df["infant"])
plt.xlabel("infant")
plt.ylabel("counts")
_ = plt.show()
plt.hist(leinhardt_df["income"])
plt.xlabel("income")
plt.ylabel("counts")
_ = plt.show()
leinhardt_df["log_income"] = np.log(leinhardt_df["income"])
leinhardt_df["log_infant"] = np.log(leinhardt_df["infant"])
plt.scatter(leinhardt_df["log_income"], leinhardt_df["log_infant"])
plt.xlabel("log_income")
plt.ylabel("log_infant")
_ = plt.show()<jupyter_output><empty_output><jupyter_text>Linear Regression (single variable) on log scaleFitting a OLS model against the log scaled data results in NaN results for the coefficients. This is because of missing values in the input or caused during the log transformation. The R `lm` call is more forgiving and will automatically deal with this and report that there are 4 missing values and that the results might be incorrect.<jupyter_code>leinhardt_df_na = leinhardt_df[(leinhardt_df["log_infant"].isna()) | 
                               (leinhardt_df["log_income"].isna())]
leinhardt_df_na.head()
leinhardt_df.dropna(subset=["log_income", "log_infant"], inplace=True)<jupyter_output><empty_output><jupyter_text>Linear Regression (single variable) on log scale after NaN removedFollowing information can be seen from the result of the linear regression:* intercept is 7.14 with standard error 0.317* coefficient of income is -0.5 with standard error 0.05* both values are large compared to their standard error (deviation) of the posterior, so they appear to be very statistically significant (P>|t|).* R-squared tells us how much of the variance is explained by the linear model.<jupyter_code>x = leinhardt_df["log_income"].values
X = np.vstack((x, np.ones(len(x)))).T

y = leinhardt_df["log_infant"].values

results = sm.OLS(y, X).fit()
results.summary()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11470230862697449, 'fraction_numerical': 0.005589307411907655, 'mean_word_length': 6.766037735849056, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 26, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '510921', 'license': 'Apache-2.0', 'n_tokens_mistral': 1424, 'n_tokens_neox': 1283, 'n_words': 358, 'path': 'techniques-and-models/w03-07a-linreg-demo.ipynb', 'repo_name': 'sujitpal/bayesian-stats-examples', 'size': '4115'}