<commit_before>---
layout: post
title: Regression with Interactions Terms - how Centering Predictors influences Main Effects
category: random
---

Centering predictors in a regression model with only main effects has no influence on the main effects. In contrast, in a regression model including interaction terms centering predictors *does* have an influence on the main effects. After getting confused by this, I read [this](https://amstat.tandfonline.com/doi/pdf/10.1080/10691898.2011.11889620) nice paper on the topic and played around with the examples in R. I summarized the resulting notes and code snippets in this blogpost.

We give an explanation on two levels:

1. By illustrating the issue with the simplest possible example
2. By showing in general how main effects are a function of the constants (e.g. means) that are substracted from predictor variables


Explanation 1: Simplest example
-----


The simplest possible example to illustrate the issue is a regression model in which variable $Y$ is a linear function of variables $X_1$, $X_2$ and their product $X_1X_2$

$$
Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_1X_2 + \epsilon,
$$
where we set $\beta_0 = 1, \beta_1 = 0.3, \beta_2 = 0.2, \beta_3 = 0.2$, and $\epsilon \sim N(0, \sigma^2)$ is a draw from a Gaussian distribution with mean zero and variance $\sigma^2$. We define the predictors $X_1, X_2$ as Gaussians with means $\mu_{X_1} = \mu_{X_2} = 1$ and $\sigma_{X_1}^{2}=\sigma_{X_2}^{2}=1$. This code samples $n = 10000$ observations from this model:

{% highlight r %}

n <- 10000
b0 <- 1; b1 <- .3; b2 <- .2; b3 <- .2
set.seed(1)
x1 <- rnorm(n, mean = 1, sd = 1)
x2 <- rnorm(n, mean = 1, sd = 1)
y <- b0 + b1 * x1 + b2 * x2 + b3 * x1 * x2 + rnorm(n)

{% endhighlight %}


**Regression models with main effects**

We first verify that centering variables indeed does not affect the main effects. To do so, we first fit the linear regression with only main effects with uncenetered predictors

{% highlight r %}
lm(y ~ x1 + x2)

Call:
lm(formula = y ~ x1 + x2)

Coefficients:
(Intercept)           x1           x2  
     0.8088       0.4983       0.4015  
{% endhighlight %}

and then with centered predictors

{% highlight r %}
x1_c <- x1 - mean(x1) # center predictors
x2_c <- x2 - mean(x2)
lm(y ~ x1_c + x2_c)

Call:
lm(formula = y ~ x1_c + x2_c)

Coefficients:
(Intercept)         x1_c         x2_c  
     1.7036       0.4983       0.4015  
{% endhighlight %}


The parameter estimates of the regression with uncentered predictors are $\hat\beta_1 \approx 0.50$ and $\hat\beta_2 \approx 0.40$. The estimates of the regression with *centered* predictors are $\hat\beta_1^* \approx 0.50$ and $\hat\beta_2^* \approx 0.40$ (we denote estimates from regressions with centered predictors with an asterisk). And indeed $\hat\beta_1 = \hat\beta_1^*$ and $\hat\beta_2 = \hat\beta_2^*$ 


are $\hat\beta_1 \approx 0.50$ and $\hat\beta_2 \approx 0.40$.

are $\hat\beta_1 \approx 0.50$ and $\hat\beta_2 \approx 0.40$.

are $\hat\beta_1^* \approx 0.50$ and

are $\hat\beta_1^* = \hat\beta_1^* $ and

are $\hat\beta_1^* = \hat\beta_1^*$ and are $\hat\beta_1^* = \hat\beta_1^*$ and

And indeed $\hat\beta_1 = \hat\beta_1^*$ and $\hat\beta_2 = \hat\beta_2^*$.

And indeed, $\hat\beta_1 = \hat\beta_1^*$ and $\hat\beta_2 = \hat\beta_2^*$ 

And indeed, $\hat\beta_1 = \hat\beta_1^*$ and $\hat\beta_2 = \hat\beta_2^*$.


**Regression models with main effects + interaction**

We include the interaction term and show that centering the predictors now does *does* affect the main effects. We first fit the regression model without centering

{% highlight r %}
lm(y ~ x1 * x2)

Call:
lm(formula = y ~ x1 * x2)

Coefficients:
(Intercept)           x1           x2        x1:x2  
     1.0183       0.2883       0.1898       0.2111  
{% endhighlight %}

and then with centering<commit_msg>add: centering v16<commit_after>---
layout: post
title: Regression with Interactions Terms - how Centering Predictors influences Main Effects
category: random
---

Centering predictors in a regression model with only main effects has no influence on the main effects. In contrast, in a regression model including interaction terms centering predictors *does* have an influence on the main effects. After getting confused by this, I read [this](https://amstat.tandfonline.com/doi/pdf/10.1080/10691898.2011.11889620) nice paper on the topic and played around with the examples in R. I summarized the resulting notes and code snippets in this blogpost.

We give an explanation on two levels:

1. By illustrating the issue with the simplest possible example
2. By showing in general how main effects are a function of the constants (e.g. means) that are substracted from predictor variables


Explanation 1: Simplest example
-----


The simplest possible example to illustrate the issue is a regression model in which variable $Y$ is a linear function of variables $X_1$, $X_2$ and their product $X_1X_2$

$$
Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_1X_2 + \epsilon,
$$
where we set $\beta_0 = 1, \beta_1 = 0.3, \beta_2 = 0.2, \beta_3 = 0.2$, and $\epsilon \sim N(0, \sigma^2)$ is a draw from a Gaussian distribution with mean zero and variance $\sigma^2$. We define the predictors $X_1, X_2$ as Gaussians with means $\mu_{X_1} = \mu_{X_2} = 1$ and $\sigma_{X_1}^{2}=\sigma_{X_2}^{2}=1$. This code samples $n = 10000$ observations from this model:

{% highlight r %}

n <- 10000
b0 <- 1; b1 <- .3; b2 <- .2; b3 <- .2
set.seed(1)
x1 <- rnorm(n, mean = 1, sd = 1)
x2 <- rnorm(n, mean = 1, sd = 1)
y <- b0 + b1 * x1 + b2 * x2 + b3 * x1 * x2 + rnorm(n)

{% endhighlight %}


**Regression models with main effects**

We first verify that centering variables indeed does not affect the main effects. To do so, we first fit the linear regression with only main effects with uncenetered predictors

{% highlight r %}
lm(y ~ x1 + x2)

Call:
lm(formula = y ~ x1 + x2)

Coefficients:
(Intercept)           x1           x2  
     0.8088       0.4983       0.4015  
{% endhighlight %}

and then with centered predictors

{% highlight r %}
x1_c <- x1 - mean(x1) # center predictors
x2_c <- x2 - mean(x2)
lm(y ~ x1_c + x2_c)

Call:
lm(formula = y ~ x1_c + x2_c)

Coefficients:
(Intercept)         x1_c         x2_c  
     1.7036       0.4983       0.4015  
{% endhighlight %}


The parameter estimates of the regression with uncentered predictors are $\hat\beta_1 \approx 0.50$ and $\hat\beta_2 \approx 0.40$. The estimates of the regression with *centered* predictors are $\hat\beta_1^* \approx 0.50$ and $\hat\beta_2^* \approx 0.40$ (we denote estimates from regressions with centered predictors with an asterisk). And indeed $$\hat\beta_1 = \hat\beta_1^*$$ and $$\hat\beta_2 = \hat\beta_2^*$$


are $\hat\beta_1 \approx 0.50$ and $\hat\beta_2 \approx 0.40$.

are $\hat\beta_1 \approx 0.50$ and $\hat\beta_2 \approx 0.40$.

are $\hat\beta_1^* \approx 0.50$ and

are $\hat\beta_1^* = \hat\beta_1^* $ and

are $$\hat\beta_1^* = \hat\beta_1^*$$ and are $$\hat\beta_1^* = \hat\beta_1^*$$ and

And indeed $\hat\beta_1 = \hat\beta_1^*$ and $\hat\beta_2 = \hat\beta_2^*$.

And indeed, $$\hat\beta_1 = \hat\beta_1^*$$ and $$\hat\beta_2 = \hat\beta_2^*$$

And indeed, $\hat\beta_1 = \hat\beta_1^*$ and $\hat\beta_2 = \hat\beta_2^*$.


**Regression models with main effects + interaction**

We include the interaction term and show that centering the predictors now does *does* affect the main effects. We first fit the regression model without centering

{% highlight r %}
lm(y ~ x1 * x2)

Call:
lm(formula = y ~ x1 * x2)

Coefficients:
(Intercept)           x1           x2        x1:x2  
     1.0183       0.2883       0.1898       0.2111  
{% endhighlight %}

and then with centering
{'fraction_non_alphanumeric': 0.1233337647211078, 'fraction_numerical': 0.060308010870971915, 'mean_word_length': 3.1886178861788617, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2387308', 'n_tokens_mistral': 2913, 'n_tokens_neox': 2603, 'n_words': 995}