Interaction models overview

What do interactions do?

Interaction models allow us to model a scenario when the association between one predictor \(x\) and the outcome \(y\) is different, depending on the level of another predictor \(z\). Mathematically, this works by adding another predictor to the model that is the product (in other words, the result of multiplying) those two predictors.

In a multiple regression model without the interaction, the effect of x on y is just \(\beta_1\), a single constant number.

\[ y = \beta_0 + (\underbrace{\beta_1}_{\text{effect of } x} \cdot x) + (\beta_2 \cdot z) + \epsilon \]

But if the effect of x depends on z, then the effect isn’t just a single constant number. It’s more like “some number plus some amount of z”.

To represent “some amount of z”, we can write \(\beta_3 \cdot z\) and add it on to \(\beta_1\), the effect of x.

\[ y = \beta_0 + (\underbrace{(\overbrace{\beta_1}^{\text{some number}} + \overbrace{\beta_3 \cdot z}^{\text{some amount of } z})}_{\text{effect of } x} \cdot x) + (\beta_2 \cdot z) + \epsilon \]

And if you expand this expression by multiplying both \(\beta_1\) and \(\beta_3 \cdot z\) by \(x\), you get the final linear expression we use for this model:

\[ y = \beta_0 + (\beta_1 \cdot x) + (\beta_2 \cdot z) + (\beta_3 \cdot x \cdot z) + \epsilon \]

Mathematical model formulation

\[ y = \beta_0 + (\beta_1 \cdot x) + (\beta_2 \cdot z) + (\beta_3 \cdot x \cdot z) + \epsilon \]

Write:

$$
y = \beta_0 + 
(\beta_1 \cdot x) + 
(\beta_2 \cdot z) + 
(\beta_3 \cdot x \cdot z) + 
\epsilon
$$

For your analyses, interaction models should always include the two predictors individually as well as their interaction.

“Except in special circumstances, a model including a product term for interaction between two explanatory variables should also include terms with each of the explanatory variables individually, even though their coefficients may not be significantly different from zero. Following this rule avoids the logical inconsistency of saying that the effect of \(X_1\) depends on the level of \(X_2\) but that there is no effect of \(X_1\).” (Ramsey, 2012)

In R

We usually write:

model_name <- lm(y ~ x * z, data = data_name)

where x * z is a shorthand for x + z + x:z, which we could also write in full:

model_name <- lm(y ~ x + z + x:z, data = data_name)

Interpreting coefficients

The interaction model illustrated above will have four coefficients: (Intercept), x, z, and x:z.

(Intercept)

The intercept has the same interpretation as always: the estimated mean outcome when all predictors are equal to zero.

x and z (slopes = conditional effects)

For each slope parameter in an interaction model, we estimate its association with the outcome specifically when the other interacting predictor is equal to zero.

This means we can no longer talk about the “effect of x holding z constant, like we would if it were just a multiple regression model. Instead we must talk about x as a conditional effect: its interpretation is conditional on (dependent on, contingent on) z being at a specific value, namely zero.

So when we fit the model

\[ y = \beta_0 + (\beta_1 \cdot x) + (\beta_2 \cdot z) + (\beta_3 \cdot x \cdot z) + \epsilon \]

  • the \(\beta_1\) is the conditional effect of \(x\) on \(y\) when \(z = 0\)
  • the \(\beta_2\) is the conditional effect of \(z\) on \(y\) when \(x = 0\)

Yes, technically speaking, all parameter estimates in multiple regression are “conditional” in the sense that they are dependent upon the inclusion of other variables in the model.

For instance, in the multiple regression model \(y = \beta_0 + \beta_1 \cdot x_1 + \beta_2 \cdot x_2 + \epsilon\), the coefficient \(\hat \beta_1\) is conditional upon holding \(x_2\) constant. But we are only going to use the term “conditional effects” for the slope coefficients of interaction models, in order to distinguish them from the slope coefficients of multiple regression models.

x:z (the interaction term)

The coefficient for an interaction term is

  • an adjustment to the conditional effect of x when z increases by 1, and at the same time,
  • an adjustment to the conditional effect of z when x increases by 1.

For example, if the interaction term is –2:

  • increasing x by 1 decreases the association between z and y by 2 units, and at the same time,
  • increasing z by 1 decreases the association between x and y by 2 units.

What about other non-interacting predictors?

The interaction x:z changes how we interpret the slope coefficients for the interacting predictors x and z. But it does not change how we interpret the slope coefficients for other predictors in our model.

Any variables that are not involved in the interaction are still just held constant. And when we interpret those non-interacting coefficients, we hold constant the interacting variables.

For example, suppose we also had another predictor \(c\) in our model:

\[ y = \beta_0 + (\beta_1 \cdot x) + (\beta_2 \cdot z) + (\beta_3 \cdot x \cdot z) + (\beta_4 \cdot c) + \epsilon \]

or equivalently in R notation

lm(y ~ x + z + x:z + c)

the interpretations are as follows:

Coefficient Interpretation
(Intercept) = \(\beta_0\) The estimated mean outcome when all predictors are zero
x = \(\beta_1\) Holding c constant, when z is zero, how much does y increase/decrease when x is increased by one unit?
z = \(\beta_2\) Holding c constant, when x is zero, how much does y increase/decrease when z is increased by one unit?
x:z = \(\beta_3\) (1) How much does the association between x and y increase/decrease when z is increased by one unit, holding c constant?
(2) How much does the association between z and y increase/decrease when x is increased by one unit, holding c constant?
c = \(\beta_4\) Holding x and z constant, how much does y increase/decrease when c is increased by one unit?