<jupyter_start><jupyter_text>Model$Y = \beta_0 + \beta_1 x_1 + \dots + \beta_k x_k + \epsilon$  Canonical X<jupyter_code>x = x_fix.copy()
e = e_fix.copy()
y = y_fix.copy()
data = pd.DataFrame({"x": x, "e": e, "y": y})

regressor = smf.ols(formula="y ~ x", data=data).fit()
print(regressor.params)

with sns.plotting_context("talk"):
    fig, ax = plt.subplots(figsize=(10, 10))
    sns.scatterplot(x="x", y="y", data=data, color=brand.NU_LIGHT_PURPLE_MATPLOT)
    sns.regplot(x="x", y="y", data=data, color=brand.NU_BRIGHT_PURPLE_MATPLOT)
    plt.show()<jupyter_output>Intercept    7.680093
x            1.288105
dtype: float64<jupyter_text>OutliersLeverage: an outlier in the X space has high leverage.  High leverage points tend to have low residuals $E_i$ because they can influence the coefficients!!! When they a in-line with the data, they also increase the confidence on the estimates/decreases variance estimates!!!Hat values $h_i$ measure it. Heuristically, an outlier has $h_i > 2\bar{h}$ or $3\bar{h}$, with $\bar{h} = (k+1) / n$ for the OLS regression.  Discrepancy: how far from the tendency is the point; an "in-line" observation has low discrepancy.  Regresion outlier: how unusual is a point on Y given X; how far from the centroid in the XY space.  Deleted studentized residual $E^*_i$: good for detecting outliers. Heuristically, an outlier has $|E^*_i| > 2$ or $3$.The influence on coefficients is `Influence = Leverage X Discrepancy`.Cook's distance: measures how influential is a point on the coefficients<jupyter_code>np.random.seed(seed)
idx = np.random.randint(0, n, 1)<jupyter_output><empty_output><jupyter_text>**Low leverage, high discrepancy**<jupyter_code>x = x_fix.copy()
e = e_fix.copy()
y = y_fix.copy()
x[idx] = -150
y[idx] = -150
data = pd.DataFrame({"x": x,
                     "e": e,
                     "y": y})

regressor = smf.ols(formula="y ~ x", data=data).fit()
print(regressor.params)

with sns.plotting_context("talk"):
    fig, ax = plt.subplots(figsize=(10, 10))
    sns.scatterplot(x="x", y="y", data=data, color=brand.NU_LIGHT_PURPLE_MATPLOT)
    sns.regplot(x="x", y="y", data=data, color=brand.NU_BRIGHT_PURPLE_MATPLOT)
    ax.axline((0, true_params["b0"]), slope=true_params["b1"], color=brand.NU_DARK_PURPLE_MATPLOT, linestyle="--")
    ax.set_xlim(x.min()-10, x.max()+10)
    plt.savefig(f"../imgs/simul_high_leverage_low_discrepancy.png", bbox_inches="tight")
    plt.show()<jupyter_output>Intercept    7.846298
x            1.280766
dtype: float64<jupyter_text>**Low leverage, high discrepancy**<jupyter_code>x = x_fix.copy()
e = e_fix.copy()
y = y_fix.copy()
x[idx] = 50
y[idx] = -350
data = pd.DataFrame({"x": x,
                     "e": e,
                     "y": y})

regressor = smf.ols(formula="y ~ x", data=data).fit()
print(regressor.params)

with sns.plotting_context("talk"):
    fig, ax = plt.subplots(figsize=(10, 10))
    sns.scatterplot(x="x", y="y", data=data, color=brand.NU_LIGHT_PURPLE_MATPLOT)
    sns.regplot(x="x", y="y", data=data, color=brand.NU_BRIGHT_PURPLE_MATPLOT)
    ax.axline((0, true_params["b0"]), slope=true_params["b1"], color=brand.NU_DARK_PURPLE_MATPLOT, linestyle="--")
    ax.set_xlim(x.min()-10, x.max()+10)
    plt.savefig(f"../imgs/simul_low_leverage_high_discrepancy.png", bbox_inches="tight")
    plt.show()<jupyter_output>Intercept    6.517064
x            1.273340
dtype: float64<jupyter_text>**High leverage, high discrepancy**<jupyter_code>x = x_fix.copy()
e = e_fix.copy()
y = y_fix.copy()
x[idx] = -150
y[idx] = 50
data = pd.DataFrame({"x": x,
                     "e": e,
                     "y": y})

regressor = smf.ols(formula="y ~ x", data=data).fit()
print(regressor.params)

with sns.plotting_context("talk"):
    fig, ax = plt.subplots(figsize=(10, 10))
    sns.scatterplot(x="x", y="y", data=data, color=brand.NU_LIGHT_PURPLE_MATPLOT)
    sns.regplot(x="x", y="y", data=data, color=brand.NU_BRIGHT_PURPLE_MATPLOT)
    ax.axline((0, true_params["b0"]), slope=true_params["b1"], color=brand.NU_DARK_PURPLE_MATPLOT, linestyle="--")
    ax.set_xlim(x.min()-10, x.max()+10)
    plt.savefig(f"../imgs/simul_high_leverage_high_discrepancy.png", bbox_inches="tight")
    plt.show()<jupyter_output>Intercept    9.986489
x            1.222041
dtype: float64
{'fraction_non_alphanumeric': 0.1647527536911179, 'fraction_numerical': 0.02976329974220764, 'mean_word_length': 4.243243243243243, 'pattern_counts': {'":': 12, '<': 17, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '37261', 'license': 'MIT', 'n_tokens_mistral': 1808, 'n_tokens_neox': 1689, 'n_words': 340, 'path': 'notebooks/Outliars.ipynb', 'repo_name': 'victordalla/regression-diagnosis', 'size': '4267'}