Week 4, part 1: Three-Level Nesting

class: center, middle, inverse, title-slide

# <b>Week 4, part 1: Three-Level Nesting</b>
## Multivariate Statistics and Methodology using R (MSMR)<br><br>
### Dan Mirman
### Department of Psychology<br>The University of Edinburgh

---

# Turtles all the way down
.pull-left[
MLM is useful for **nested data**.

So far, we've considered two levels of nesting:

* Group level
* Individual level

It's also possible to have multiple nesting levels or other kinds of crossed nesting structures.

This is captured by the **random effect structure**.
]

.pull-right[
<img src="./figs/turtles_wikipedia.jpg" />

image credit: wikipedia
]

---
# 3-level nesting

.left-column[
**Sources of variability (random intercepts and time-slopes)**

* Therapists 
* Subjects
]

.right-column[
<img src="./figs/3lvl_rpsychologist.png" />

image credit: Kristoffer Magnusson
]

---
# Example: Math Active Learning
**Simulated example: Testing whether a new (computer based) active learning method improves math test scores.**

* You measured students' math scores (DV) and the proportion of time (IV) they spent using the computer. 
* The program was implemented in 3 schools
* 8-12 classrooms in each school (total schools n=30)
* 12-24 students per class (total students n=570)

image credit: Alexander P. Demos

---
# Math Active Learning: Read the data

Do a little data wrangling: center some values relative to the grand mean and the school mean.

```r
math <- read_csv("./data/active_math_sim.csv") %>%
  mutate(Class = paste0(School, "cl", Classroom), #unique class ID
         ActiveTime.GM = scale(ActiveTime, scale=F), #grand-mean centered
         ClassSize.GM = scale(ClassSize, scale=F)) %>% #grand-mean centered
  group_by(School) %>%
  mutate(ClassSize.SM = scale(ClassSize, scale=F)) #school-mean centered
```

```
## Rows: 570 Columns: 6
```

```
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (1): School
## dbl (5): Math, ActiveTime, ClassSize, Classroom, StudentID
```

```
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
```

---
# Math Active Learning: Exploratory plot 1

```r
p1a <- ggplot(math, aes(ClassSize, Math)) + geom_point() + geom_smooth(method = "lm")
p1b <- ggplot(math, aes(ActiveTime, Math)) + geom_point() + geom_smooth(method = "lm")
p1a + p1b # requires "patchwork" library
```

![](msmr_lec04-1_RanEf_3lev_files/figure-html/unnamed-chunk-2-1.png)

Looks like math scores are better in **smaller classes** and for students with a higher **active learning proportion**.

---
# Math Active Learning: Exploratory plot 2

```r
p2a <- ggplot(math, aes(Class, ClassSize, color=School)) + geom_point() + coord_flip()
p2b <- ggplot(math, aes(Class, ActiveTime, fill=School)) + geom_boxplot() + coord_flip()
p2a + p2b
```

![](msmr_lec04-1_RanEf_3lev_files/figure-html/unnamed-chunk-3-1.png)

* Classes differ in size, and this is unequal across schools
* Students differ in proportion  of time spent on active learning, but each class has a big (similar) range

---
# Math Active Learning: 2-level model

```r
m1 <- lmer(Math ~ ActiveTime.GM * ClassSize.GM + (1 | Class),  
          data=math, REML=FALSE)
gt(tidy(m1)) # requires "broom.mixed" and "gt" packages
```

<div id="hhjxciztch" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#hhjxciztch .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#hhjxciztch .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#hhjxciztch .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#hhjxciztch .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 4px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#hhjxciztch .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#hhjxciztch .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#hhjxciztch .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#hhjxciztch .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#hhjxciztch .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#hhjxciztch .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#hhjxciztch .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#hhjxciztch .gt_group_heading {
  padding: 8px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#hhjxciztch .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#hhjxciztch .gt_from_md > :first-child {
  margin-top: 0;
}

#hhjxciztch .gt_from_md > :last-child {
  margin-bottom: 0;
}

#hhjxciztch .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#hhjxciztch .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 12px;
}

#hhjxciztch .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#hhjxciztch .gt_first_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
}

#hhjxciztch .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#hhjxciztch .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#hhjxciztch .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#hhjxciztch .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#hhjxciztch .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#hhjxciztch .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding: 4px;
}

#hhjxciztch .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#hhjxciztch .gt_sourcenote {
  font-size: 90%;
  padding: 4px;
}

#hhjxciztch .gt_left {
  text-align: left;
}

#hhjxciztch .gt_center {
  text-align: center;
}

#hhjxciztch .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#hhjxciztch .gt_font_normal {
  font-weight: normal;
}

#hhjxciztch .gt_font_bold {
  font-weight: bold;
}

#hhjxciztch .gt_font_italic {
  font-style: italic;
}

#hhjxciztch .gt_super {
  font-size: 65%;
}

#hhjxciztch .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 65%;
}
</style>
<table class="gt_table">
  
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">effect</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">group</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">term</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">estimate</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">std.error</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">statistic</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">df</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">p.value</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">(Intercept)</td>
<td class="gt_row gt_right">42.9645</td>
<td class="gt_row gt_right">1.5858</td>
<td class="gt_row gt_right">27.093</td>
<td class="gt_row gt_right">29.93</td>
<td class="gt_row gt_right">1.291e-22</td></tr>
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">ActiveTime.GM</td>
<td class="gt_row gt_right">14.9057</td>
<td class="gt_row gt_right">0.6044</td>
<td class="gt_row gt_right">24.662</td>
<td class="gt_row gt_right">540.66</td>
<td class="gt_row gt_right">1.509e-90</td></tr>
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">ClassSize.GM</td>
<td class="gt_row gt_right">-1.8662</td>
<td class="gt_row gt_right">0.4805</td>
<td class="gt_row gt_right">-3.884</td>
<td class="gt_row gt_right">30.05</td>
<td class="gt_row gt_right">5.244e-04</td></tr>
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">ActiveTime.GM:ClassSize.GM</td>
<td class="gt_row gt_right">0.3613</td>
<td class="gt_row gt_right">0.1939</td>
<td class="gt_row gt_right">1.863</td>
<td class="gt_row gt_right">540.59</td>
<td class="gt_row gt_right">6.300e-02</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">sd__(Intercept)</td>
<td class="gt_row gt_right">8.5091</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Residual</td>
<td class="gt_row gt_left">sd__Observation</td>
<td class="gt_row gt_right">4.1709</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
  </tbody>
  
  
</table>
</div>

As we work through this example, keep an eye on the effect of class size and its interaction with active learning time.

---
# Math Active Learning: 2-level model fit(s)

```r
ggplot(augment(m1), aes(ActiveTime.GM, Math, color=Class)) +
  geom_point() + geom_smooth(aes(y=.fitted), method = "lm") +
  xlab("Active Learning Time") + ylab("Math Score") + theme(legend.position = "none")
```

![](msmr_lec04-1_RanEf_3lev_files/figure-html/unnamed-chunk-5-1.png)

Note the parallel lines: Our model allows classrooms to differ in math scores (random intercept), but not in the effect of active learning, which is assumed to be the same in all classrooms.

Maybe classes differ in how strongly active learning influences math achievement? **Add random slopes**

---
# Math Active Learning: 2-level model, part 2

```r
m2 <- lmer(Math ~ ActiveTime.GM * ClassSize.GM + (1 + ActiveTime.GM | Class), data=math, REML=F)
```
<div id="kcoihmyjci" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#kcoihmyjci .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#kcoihmyjci .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#kcoihmyjci .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#kcoihmyjci .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 4px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#kcoihmyjci .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#kcoihmyjci .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#kcoihmyjci .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#kcoihmyjci .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#kcoihmyjci .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#kcoihmyjci .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#kcoihmyjci .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#kcoihmyjci .gt_group_heading {
  padding: 8px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#kcoihmyjci .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#kcoihmyjci .gt_from_md > :first-child {
  margin-top: 0;
}

#kcoihmyjci .gt_from_md > :last-child {
  margin-bottom: 0;
}

#kcoihmyjci .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#kcoihmyjci .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 12px;
}

#kcoihmyjci .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#kcoihmyjci .gt_first_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
}

#kcoihmyjci .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#kcoihmyjci .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#kcoihmyjci .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#kcoihmyjci .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#kcoihmyjci .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#kcoihmyjci .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding: 4px;
}

#kcoihmyjci .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#kcoihmyjci .gt_sourcenote {
  font-size: 90%;
  padding: 4px;
}

#kcoihmyjci .gt_left {
  text-align: left;
}

#kcoihmyjci .gt_center {
  text-align: center;
}

#kcoihmyjci .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#kcoihmyjci .gt_font_normal {
  font-weight: normal;
}

#kcoihmyjci .gt_font_bold {
  font-weight: bold;
}

#kcoihmyjci .gt_font_italic {
  font-style: italic;
}

#kcoihmyjci .gt_super {
  font-size: 65%;
}

#kcoihmyjci .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 65%;
}
</style>
<table class="gt_table">
  
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">effect</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">group</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">term</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">estimate</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">std.error</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">statistic</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">df</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">p.value</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">(Intercept)</td>
<td class="gt_row gt_right">42.8612</td>
<td class="gt_row gt_right">1.5984</td>
<td class="gt_row gt_right">26.815</td>
<td class="gt_row gt_right">29.95</td>
<td class="gt_row gt_right">1.715e-22</td></tr>
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">ActiveTime.GM</td>
<td class="gt_row gt_right">14.8927</td>
<td class="gt_row gt_right">1.0450</td>
<td class="gt_row gt_right">14.251</td>
<td class="gt_row gt_right">30.32</td>
<td class="gt_row gt_right">5.614e-15</td></tr>
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">ClassSize.GM</td>
<td class="gt_row gt_right">-1.8703</td>
<td class="gt_row gt_right">0.4843</td>
<td class="gt_row gt_right">-3.862</td>
<td class="gt_row gt_right">30.04</td>
<td class="gt_row gt_right">5.561e-04</td></tr>
    <tr><td class="gt_row gt_left" style="background-color: #FFFF00;">fixed</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">NA</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">ActiveTime.GM:ClassSize.GM</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.3329</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.3263</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">1.020</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">33.11</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">3.150e-01</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">sd__(Intercept)</td>
<td class="gt_row gt_right">8.5821</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">cor__(Intercept).ActiveTime.GM</td>
<td class="gt_row gt_right">-0.3107</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">sd__ActiveTime.GM</td>
<td class="gt_row gt_right">4.7109</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
  </tbody>
  
  
</table>
</div>

Adding `ActiveTime` random slopes substantially increased the SE and reduced estimated df for `ActiveTime` fixed effects. Interaction effect is not even marginal in this model: in the first model, omitting the random slope made the corresponding fixed effect anti-conservative.

Now that we have that random slope, the model can capture *random* classroom-level variation in effect of active learning instead of spuriously assigning it to class.

---
#  Math Active Learning: 2-level model, part 2 fit(s)

```r
ggplot(augment(m2), aes(ActiveTime.GM, Math, color=Class)) +
  geom_point() + geom_smooth(aes(y=.fitted), method = "lm") +
  xlab("Active Learning Time") + ylab("Math Score") + theme(legend.position = "none")
```

![](msmr_lec04-1_RanEf_3lev_files/figure-html/unnamed-chunk-8-1.png)

Model captures random class-level variation in math scores and strength of active learning effect.

---
#  Math Active Learning: a 3-level model?

The 2-level model assumes that classrooms are independent, but this is not quite true. Classrooms are nested within schools and there may be school-level differences in math scores, effects of active learning, class size, etc.

```r
ggplot(math, aes(ActiveTime.GM, Math, color=Class)) +
  facet_wrap(~ School) +
  geom_point() + geom_smooth(method = "lm", se=FALSE) +
  xlab("Active Learning Time") + ylab("Math Score") + theme(legend.position = "none")
```

![](msmr_lec04-1_RanEf_3lev_files/figure-html/unnamed-chunk-9-1.png)

---
##  School and Class Size are confounded

```r
unique(select(math, School, Class, ClassSize, ClassSize.GM, ClassSize.SM)) %>%
  pivot_longer(cols=starts_with("ClassSize"), names_to = "Type", values_to = "Size") %>%
  ggplot(aes(School, Size)) + facet_wrap(~ Type, scales="free_y") + geom_violin() + geom_jitter(width=0.1, height=0)
```

![](msmr_lec04-1_RanEf_3lev_files/figure-html/unnamed-chunk-10-1.png)

* Class sizes are different across schools, which means that these variables are confounded: can't distinguish (random, uninteresting) differences between schools from (systematic, interesting) effects of class size.
* This is true for both raw and grand-mean-centered class sizes
* School-mean-centered class sizes allow testing effect of class size while controlling for school-level differences

---
# Sidebar: Simpson's Paradox

.pull-left[
<img src="./figs/Simpsons_paradox.png" />
]

.pull-right[
**Famous real-world example: gender bias in PG admissions at University of California - Berkeley in 1973**

* Overall acceptance rate higher for males than females
* But individual department acceptance rates higher for females than males

This is usually because of a "lurking" (confounding) variable. In the Berkeley example there is a confound between gender differences in **application** rates and **department** differences in acceptance rates: departments with low admission rates had many more female applicants, departments with higher admission rates had many more male applicants. As a result, even though (almost) all departments had higher acceptance rates for females, the *overall* acceptance rate was higher for males.
]

**Lesson**: watch out for sub-group (cluster-level) differences that might be confounded with your variables of interest.

---
#  Math Active Learning: 3-level model

* Use school-mean-centered measure of class size to test effect of class size while controlling for school-level differences
* Add random effect of School (intercept and slope)

```r
m3 <- lmer(Math ~ ActiveTime.GM * ClassSize.SM +
           (1 + ActiveTime.GM | School) +
           (1 + ActiveTime.GM | Class),  
           data=math, REML=FALSE)
```

---
#  Math Active Learning: 3-level model

<div id="zynvipyzjj" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#zynvipyzjj .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#zynvipyzjj .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#zynvipyzjj .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#zynvipyzjj .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 4px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#zynvipyzjj .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#zynvipyzjj .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#zynvipyzjj .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#zynvipyzjj .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#zynvipyzjj .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#zynvipyzjj .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#zynvipyzjj .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#zynvipyzjj .gt_group_heading {
  padding: 8px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#zynvipyzjj .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#zynvipyzjj .gt_from_md > :first-child {
  margin-top: 0;
}

#zynvipyzjj .gt_from_md > :last-child {
  margin-bottom: 0;
}

#zynvipyzjj .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#zynvipyzjj .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 12px;
}

#zynvipyzjj .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#zynvipyzjj .gt_first_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
}

#zynvipyzjj .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#zynvipyzjj .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#zynvipyzjj .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#zynvipyzjj .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#zynvipyzjj .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#zynvipyzjj .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding: 4px;
}

#zynvipyzjj .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#zynvipyzjj .gt_sourcenote {
  font-size: 90%;
  padding: 4px;
}

#zynvipyzjj .gt_left {
  text-align: left;
}

#zynvipyzjj .gt_center {
  text-align: center;
}

#zynvipyzjj .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#zynvipyzjj .gt_font_normal {
  font-weight: normal;
}

#zynvipyzjj .gt_font_bold {
  font-weight: bold;
}

#zynvipyzjj .gt_font_italic {
  font-style: italic;
}

#zynvipyzjj .gt_super {
  font-size: 65%;
}

#zynvipyzjj .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 65%;
}
</style>
<table class="gt_table">
  
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">effect</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">group</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">term</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">estimate</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">std.error</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">statistic</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">df</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">p.value</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">(Intercept)</td>
<td class="gt_row gt_right">44.45181</td>
<td class="gt_row gt_right">4.6667</td>
<td class="gt_row gt_right">9.52538</td>
<td class="gt_row gt_right">3.046</td>
<td class="gt_row gt_right">0.002303</td></tr>
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">ActiveTime.GM</td>
<td class="gt_row gt_right">14.34850</td>
<td class="gt_row gt_right">1.8121</td>
<td class="gt_row gt_right">7.91797</td>
<td class="gt_row gt_right">3.233</td>
<td class="gt_row gt_right">0.003193</td></tr>
    <tr><td class="gt_row gt_left" style="background-color: #FFFF00;">fixed</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">NA</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">ClassSize.SM</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">-0.03656</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.5729</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">-0.06381</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">27.971</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.949572</td></tr>
    <tr><td class="gt_row gt_left" style="background-color: #FFFF00;">fixed</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">NA</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">ActiveTime.GM:ClassSize.SM</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">-0.78320</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.4133</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">-1.89499</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">32.516</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.067014</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">sd__(Intercept)</td>
<td class="gt_row gt_right">6.74413</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">cor__(Intercept).ActiveTime.GM</td>
<td class="gt_row gt_right">0.12463</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">sd__ActiveTime.GM</td>
<td class="gt_row gt_right">3.60253</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">School</td>
<td class="gt_row gt_left">sd__(Intercept)</td>
<td class="gt_row gt_right">7.78136</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">School</td>
<td class="gt_row gt_left">cor__(Intercept).ActiveTime.GM</td>
<td class="gt_row gt_right">-1.00000</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">School</td>
<td class="gt_row gt_left">sd__ActiveTime.GM</td>
<td class="gt_row gt_right">2.74252</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Residual</td>
<td class="gt_row gt_left">sd__Observation</td>
<td class="gt_row gt_right">3.91469</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
  </tbody>
  
  
</table>
</div>

Effect of Class Size is no longer significant: it was a Simpson's paradox effect (differences between Schools, which are confounded with class size, not effects of class size)

---
#  Math Active Learning: 3-level model

.pull-left[
With only 3 schools, the school-level random effects are probably over-parameterized...

```r
VarCorr(m3)
```

```
##  Groups   Name          Std.Dev. Corr 
##  Class    (Intercept)   6.74          
##           ActiveTime.GM 3.60     0.12 
##  School   (Intercept)   7.78          
##           ActiveTime.GM 2.74     -1.00
##  Residual               3.91
```

Yes, the slope-intercept correlation is -1.00

]

.pull-right[
Remove the by-school slope-intercept correlation:

```r
m3a <- lmer(Math ~ ActiveTime.GM * ClassSize.SM +
           (1 + ActiveTime.GM || School) +
           (1 + ActiveTime.GM | Class),  
           data=math, REML=FALSE)
VarCorr(m3a)
```

```
##  Groups   Name          Std.Dev. Corr
##  Class    (Intercept)   6.79         
##           ActiveTime.GM 3.76     0.14
##  School   ActiveTime.GM 2.56         
##  School.1 (Intercept)   7.88         
##  Residual               3.91
```
]
---
#  Math Active Learning: 3-level model
<div id="tmkdvcaofo" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#tmkdvcaofo .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#tmkdvcaofo .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#tmkdvcaofo .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#tmkdvcaofo .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 4px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#tmkdvcaofo .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#tmkdvcaofo .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#tmkdvcaofo .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#tmkdvcaofo .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#tmkdvcaofo .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#tmkdvcaofo .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#tmkdvcaofo .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#tmkdvcaofo .gt_group_heading {
  padding: 8px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#tmkdvcaofo .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#tmkdvcaofo .gt_from_md > :first-child {
  margin-top: 0;
}

#tmkdvcaofo .gt_from_md > :last-child {
  margin-bottom: 0;
}

#tmkdvcaofo .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#tmkdvcaofo .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 12px;
}

#tmkdvcaofo .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#tmkdvcaofo .gt_first_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
}

#tmkdvcaofo .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#tmkdvcaofo .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#tmkdvcaofo .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#tmkdvcaofo .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#tmkdvcaofo .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#tmkdvcaofo .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding: 4px;
}

#tmkdvcaofo .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#tmkdvcaofo .gt_sourcenote {
  font-size: 90%;
  padding: 4px;
}

#tmkdvcaofo .gt_left {
  text-align: left;
}

#tmkdvcaofo .gt_center {
  text-align: center;
}

#tmkdvcaofo .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#tmkdvcaofo .gt_font_normal {
  font-weight: normal;
}

#tmkdvcaofo .gt_font_bold {
  font-weight: bold;
}

#tmkdvcaofo .gt_font_italic {
  font-style: italic;
}

#tmkdvcaofo .gt_super {
  font-size: 65%;
}

#tmkdvcaofo .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 65%;
}
</style>
<table class="gt_table">
  
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">effect</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">group</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">term</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">estimate</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">std.error</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">statistic</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">df</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">p.value</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">(Intercept)</td>
<td class="gt_row gt_right">44.35577</td>
<td class="gt_row gt_right">4.7261</td>
<td class="gt_row gt_right">9.38533</td>
<td class="gt_row gt_right">3.033</td>
<td class="gt_row gt_right">0.002451</td></tr>
    <tr><td class="gt_row gt_left">fixed</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">ActiveTime.GM</td>
<td class="gt_row gt_right">14.32683</td>
<td class="gt_row gt_right">1.7365</td>
<td class="gt_row gt_right">8.25056</td>
<td class="gt_row gt_right">3.177</td>
<td class="gt_row gt_right">0.003003</td></tr>
    <tr><td class="gt_row gt_left" style="background-color: #FFFF00;">fixed</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">NA</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">ClassSize.SM</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">-0.02766</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.5772</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">-0.04792</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">27.226</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.962128</td></tr>
    <tr><td class="gt_row gt_left" style="background-color: #FFFF00;">fixed</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">NA</td>
<td class="gt_row gt_left" style="background-color: #FFFF00;">ActiveTime.GM:ClassSize.SM</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">-0.77926</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.4231</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">-1.84189</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">29.967</td>
<td class="gt_row gt_right" style="background-color: #FFFF00;">0.075409</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">sd__(Intercept)</td>
<td class="gt_row gt_right">6.79290</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">cor__(Intercept).ActiveTime.GM</td>
<td class="gt_row gt_right">0.13698</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Class</td>
<td class="gt_row gt_left">sd__ActiveTime.GM</td>
<td class="gt_row gt_right">3.75530</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">School</td>
<td class="gt_row gt_left">sd__ActiveTime.GM</td>
<td class="gt_row gt_right">2.55827</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">School.1</td>
<td class="gt_row gt_left">sd__(Intercept)</td>
<td class="gt_row gt_right">7.88169</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
    <tr><td class="gt_row gt_left">ran_pars</td>
<td class="gt_row gt_left">Residual</td>
<td class="gt_row gt_left">sd__Observation</td>
<td class="gt_row gt_right">3.91480</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td>
<td class="gt_row gt_right">NA</td></tr>
  </tbody>
  
  
</table>
</div>

---
# Key points

Nested structure can extend to 3 (or more) levels and this should be captured in the random effects structure of a multilevel model

This is particularly important when sub-group (cluster-level) differences might be confounded with variables of interest (Simpson's Paradox).

Omitting random slopes tends to make models anti-conservative (inflates rates of false positives). **Recommendation:** start with a "full" or "maximal" random effect structure and reduce as needed

When there are only a few observational units (e.g., 3 schools), estimating random effects is very difficult (esp. random slopes) and can produce unreliable fixed effect estimates. Over-parameterised random effect structures should be simplified.