Welcome to the SampleSpace! 
The SampleSpace is a hidden collection of bits and pieces. Everything here is beyond optional for the USMR course.
If you’re feeling inquisitive, have a nose around.
If you came here by accident and don’t like the look of it, that is absolutely fine, you can completely forget about this page (and will probably be better off for doing so!)
simulating multiple regression
To simulate data that can be modelled with simple regression, we essentially did:
- simulate predictor X
- use X to simulate predictor Y
When we have multiple predictors, we will need to have more steps, depending on how we consider them to fit into the process.
Let’s suppose we are interested in estimating if coffee increases reaction times, so we give 100 people coffee and 100 people water, and then measure their reaction time RT.
In this design, we’re randomly allocating people to either coffee or water, so we can use sample().
And for our fake data, we’re going to suppose that drinking coffee speeds up reaction times by 100ms compared to water.
Note that how we create the RT variable is a straightforward simple regression model: \(RT = 600 - 100*(hasCoffee) + \epsilon\).
If we want to represent the full data generating process here, we could write:
\[ \begin{align} \text{RT}_i &\sim N(\mu_i, 50^2) \\ \mu_i &= 600 - 100 \cdot \text{Coffee}_i \\ \text{Coffee}_i &\sim \text{Bernoulli}(0.5) \end{align} \]
An appropriate model to estimate how coffee influences reaction times, from this very simple experimental design, would be just lm(RT ~ condition).
Fitting this model provides us with an estimate that is close to how we generated the data (we said that coffee leads to 100ms faster RT than water):
Now let’s suppose that instead of randomly allocating people into either drinking coffee or water, we gave all 200 people the choice. They came into a room, were given the choice of water or coffee, and then half an hour later we measured their reaction times.
For our fake data, we’re going to suppose exactly the same as we did before - that drinking coffee speeds up reaction times by 100ms compared to water.
In this scenario, however, the code sample(c("water","coffee"), 200, ...) doesn’t really reflect how people end up being in either the “coffee” group or the “water” group. For one example, it’s quite possible that people who are more tired will be more likely to drink coffee!
Not only that, but those tired people will probably have lower RTs!
We could generate such data as:
And we could write this as:
\[ \begin{align} RT_i &\sim \text{Normal}(\mu_i, 50^2) \\ \mu_i &= 600 - 100 \cdot \text{Coffee}_i + 200 \cdot \text{Tired}_i \\ \text{Coffee}_i &\sim \text{Bernoulli}(p_i) \\ p_i &= 0.2 + 0.6 \cdot \text{Tired}_i \\ \text{Tired}_i &\sim \text{Bernoulli}(0.4) \end{align} \]
Note that because our design has changed, the model that previously was appropriate for our first design (randomly giving people coffee/water), is no longer estimating how coffee changes reaction times (coffee results in 100ms faster RT than water).
In fact, this model might actually now say that coffee leads to slower reaction times!
Why? Because comparing the two groups (“coffee” vs “water”) is no longer twp groups that only differ with respect to what they drank. We’re actually comparing lots of tired people (in the “coffee” group) from lots of not-tired people (in the “water” group):
When we add in tired as a predictor, it estimates the effect of coffee on reaction times within the tired / not-tired people (and assumes that this effect is the same for both):
We can manually calculate the coefficient by calculating the coffee v water RT difference in both the tired and non-tired people, and then weighting them by how balanced the coffee/water is for each type of person.
This means that if we people fell into these groups:
condition
tired coffee water
not tired 50 50
tired 9 1
Then the \(\text{RT}_\text{coffee} - \text{RT}_\text{water}\) difference for the tired group would be weighted by \(\frac{9*1}{10}=0.9\) and the \(\text{RT}_\text{coffee} - \text{RT}_\text{water}\) difference for the not-tired group would be weighted \(\frac{50*50}{100}=25\), so our coefficient would more closely match what happens in the not-tired group. Which makes sense, because the comparison there is much clearer - we have 50 coffee vs 50 water!
If tired was a continuous variable, then this same logic would happen: weighting the effect of coffee in each stratum of ‘tiredness’ based on the balance of coffee/water, but cutting up the tiredness variable into infinitely many bins. Some of the bins would have no data in them, and would contribute zero weight, etc.
but…
Of course, this assumes we actually thought to measure people’s tiredness! And it assumes that there’s not things other than tiredness that are also biasing our estimates.
This is in large part why observational research is so difficult. Our questions are often wanting to capture something explanatory (i.e. “does drinking coffee change RT?”, and “by how much?”). But in order for our model to estimate our desired target, we need to capture and adjust for the other variables that could lead us astray.
For an example, here is some code to generate a more complex picture of reality. But, we’ve used select() at the end to mimick what would happen if we never measured some of the variables.
As before, drinking coffee speeds up reaction times by 100ms compared to water.
But can we see that from our models?