Multi-level Model Research Questions

class: center, middle, inverse, title-slide

.title[
# <b>Multi-level Model Research Questions</b>
]
.subtitle[
## Data Analysis for Psychology in R 3
]
.author[
### Josiah King
]
.institute[
### Department of Psychology<br/>The University of Edinburgh
]

---

class: inverse, center, middle

<h2>Part 1: Model Specification</h2>
<h2 style="text-align: left;opacity:0.3;">Part 2: Model fitting</h2>
<h2 style="text-align: left;opacity:0.3;">Part 3: Interpretation & Inference</h2>
<h2 style="text-align: left;opacity:0.3;">Part 4: Reporting</h2>

---
class: center, middle
# How do we get from research questions and study design to model specification?

---
# Study 
In 2010 A US state's commissioner for education was faced with growing community concern about rising levels of adolescent antisocial behaviours.

After a series of focus groups, the commissioner approved the trialing of an intervention in which yearly Parent Management Training (PMT) group sessions were offered to the parents of a cohort of students entering 10 different high schools. Every year, the parents were asked to fill out an informant-based version of the Aggressive Behaviour Scale (ABS), measuring verbal and physical abuse, socially inappropriate behavior, and resisting care. Where possible, the same parents were followed up throughout the child's progression through high school. Alongside this, parents from a cohort of students entering 10 further high schools in the state were recruited to also complete the same informant-based ABS, but were not offered the PMT group sessions.  
The commissioner has two main questions: **Does the presentation of aggressive behaviours increase as children enter the secondary school system? If so, is there any evidence for the effectiveness of Parent Management Training (PMT) group sessions in curbing the rise of aggressive behaviors during a child's transition into the secondary school system?**

The data is available at https://uoepsy.github.io/data/abs_intervention.csv

---
# Data

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> variable </th>
   <th style="text-align:left;"> description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> schoolid </td>
   <td style="text-align:left;"> Name of school </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ppt </td>
   <td style="text-align:left;"> Participant number </td>
  </tr>
  <tr>
   <td style="text-align:left;"> age </td>
   <td style="text-align:left;"> Age of participant (in years) at observation </td>
  </tr>
  <tr>
   <td style="text-align:left;"> interv </td>
   <td style="text-align:left;"> Informant-based Aggressive Behaviour Scale (ABS) score (range 0 to 100) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ABS </td>
   <td style="text-align:left;"> Whether or not the school was part of the intervention group in which Parent Management Training (PMT) group sessions were offered to the parents </td>
  </tr>
</tbody>
</table>

```r
absint<-read_csv("https://uoepsy.github.io/data/abs_intervention.csv") %>% mutate(interv=factor(interv))
head(absint)
```

```
## # A tibble: 6 × 5
##   schoolid                 ppt   age interv   ABS
##   <chr>                  <dbl> <dbl> <fct>  <dbl>
## 1 Blue River High School     1    12 1       47.5
## 2 Blue River High School     1    13 1       47.4
## 3 Blue River High School     1    14 1       53.0
## 4 Blue River High School     1    15 1       70.7
## 5 Blue River High School     1    16 1       66.3
## 6 Blue River High School     1    17 1       NA
```

---
# Outcome and Fixed Effects

.br2.f4.gray.bg-white[
(g)lmer(**outcome** ~ fixed effects + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- What are we interested in explaining/predicting?

]

.pull-right[
> Does the presentation of aggressive behaviours increase as children enter the secondary school system? If so, is there any evidence for the effectiveness of Parent Management Training (PMT) group sessions in curbing the rise of aggressive behaviors during a child's transition into the secondary school system?

]

---
# Outcome and Fixed Effects

.br2.f4.gray.bg-white[
(g)lmer(**outcome** ~ fixed effects + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- What are we interested in explaining/predicting?

]

.pull-right[
> 1. Does the presentation of aggressive behaviours increase as children enter the secondary school system? 
> 2. If so, is there any evidence for the effectiveness of Parent Management Training (PMT) group sessions in curbing the rise of aggressive behaviors during a child's transition into the secondary school system?

]

---
count:false
# Outcome and Fixed Effects

.br2.f4.gray.bg-white[
**(g)lmer**(outcome ~ fixed effects + (random effects | grouping structure), **family = error distribution**)
]

.pull-left[

- What are we interested in explaining/predicting?

- How is this measured?

]
.pull-right[
> 1. Does the presentation of aggressive behaviours increase as children enter the secondary school system? 
> 2. If so, is there any evidence for the effectiveness of Parent Management Training (PMT) group sessions in curbing the rise of aggressive behaviors during a child's transition into the secondary school system?

![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-3-1.svg)

]

---
count:false
# Outcome and Fixed Effects

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ **fixed effects** + (random effects | grouping structure), family = error distribution)
]

.pull-left[

- What are we interested in explaining/predicting?

- How is this measured?

- What variables are we interested in explaining this by?

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> schoolid </th>
   <th style="text-align:left;"> ppt </th>
   <th style="text-align:left;"> age </th>
   <th style="text-align:left;"> interv </th>
   <th style="text-align:left;"> ABS </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 12 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.5 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 13 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 14 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 53 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 15 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 70.7 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
  </tr>
</tbody>
</table>

]

---
# Within & Between Effects

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ **fixed effects** + (random effects | grouping structure), family = error distribution)
]

.pull-left[

- Are our questions about the effects of our predictors specifically in reference to _group means_ of predictors?

- "the effect of being higher on `$x$` *__for a__* group"  
  
  - "the effect of a group being *__on average__* higher on `$x$`"

]

---
# The Grouping Structure

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

]
--
.pull-right[
<table>
 <thead>
  <tr>
   <th style="text-align:left;"> schoolid </th>
   <th style="text-align:left;"> ppt </th>
   <th style="text-align:left;"> age </th>
   <th style="text-align:left;"> interv </th>
   <th style="text-align:left;"> ABS </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 12 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.5 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 13 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 14 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 53 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 15 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 70.7 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
  </tr>
</tbody>
</table>
]

---
# The Grouping Structure

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- Of the different ways we can group our data, which groupings are of specific inferential interest?

- Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ **fixed effects** + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- **Of the different ways we can group our data, which groupings are of specific inferential interest?**

- Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | **grouping structure**), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- Of the different ways we can group our data, which groupings are of specific inferential interest?

- **Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?**

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | **grouping structure**), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- Of the different ways we can group our data, which groupings are of specific inferential interest?

- Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?

- Is there more than one grouping of this sort, and if so, are these groupings nested? Are the labels unique?  
  
  - For each level, how many groups have we sampled?  
]
--
.pull-right[

```r
nrow(absint)
```

```
## [1] 1568
```

```r
absint %>% 
  mutate(schoolppt = interaction(schoolid,ppt)) %>%
  summarise(across(everything(), n_distinct))
```

```
## # A tibble: 1 × 6
##   schoolid   ppt   age interv   ABS schoolppt
##      <int> <int> <int>  <int> <int>     <int>
## 1       20    16     8      2   993       196
```

]
---
count: false
# The Grouping Structure

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | **grouping structure**), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- Of the different ways we can group our data, which groupings are of specific inferential interest?

- Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?

- Is there more than one grouping of this sort, and if so, are these groupings nested? Are the labels unique?  
  
  - For each level, how many groups have we sampled?  
]
.pull-right[

```r
table(absint$schoolid, absint$ppt)
```

```
##                                  
##                                   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
##   Blue River High School          8 8 8 8 8 8 8 8 8  8  0  0  0  0  0  0
##   Central Grammar School          8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Central High                    8 8 8 8 8 8 8 8 8  0  0  0  0  0  0  0
##   Clearwater Charter School       8 8 8 8 8 8 8 8 8  8  8  8  8  8  8  0
##   Clearwater School of Fine Arts  8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Coral Coast Institute           8 8 8 8 8 8 8 8 8  0  0  0  0  0  0  0
##   Eagle Mountain Technical School 8 8 8 8 8 8 8 0 0  0  0  0  0  0  0  0
##   Eastwood Institute              8 8 8 8 8 8 8 8 8  8  0  0  0  0  0  0
##   Edgewood Grammar School         8 8 8 8 8 8 8 8 8  8  0  0  0  0  0  0
##   Elk Grove School                8 8 8 8 8 8 8 8 0  0  0  0  0  0  0  0
##   Grand Mountain Elementary       8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Horizon Elementary              8 8 8 8 8 0 0 0 0  0  0  0  0  0  0  0
##   Liberty High                    8 8 8 8 8 8 8 8 0  0  0  0  0  0  0  0
##   Long Beach School               8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Oak Park Technical School       8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Seal Bay Middle School          8 8 8 8 8 8 8 8 8  0  0  0  0  0  0  0
##   South Fork Conservatory         8 8 8 8 8 8 8 8 0  0  0  0  0  0  0  0
##   Tranquillity Elementary         8 8 8 8 8 8 0 0 0  0  0  0  0  0  0  0
##   Tranquillity Middle School      8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Woodside University             8 8 8 8 8 8 8 8 8  8  8  8  8  8  8  8
```

]
---
# Random Intercepts and Slopes

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (**random effects** | grouping structure), family = error distribution)
]

.pull-left[

- Which of our fixed effects can vary for our random groups?

- "does a single group have multiple _distinct_ values for `$x$`?"  
  
  - "for the data from only one of our groups, can we estimate `$y \sim x$`?"

]
--
.pull-right[
All the data:
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-11-1.svg)
Data from School == "Central High":
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-12-1.svg)

]

---
# Random Intercepts and Slopes

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (**random effects** | grouping structure), family = error distribution)
]

.pull-left[

- Which of our fixed effects can vary for our random groups?

- "does a single group have multiple values for `$x$`?"  
  
  - "for the data from only one of our groups, can we estimate `$y \sim x$`?"

]
.pull-right[
All the data:
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-13-1.svg)
Data from School == "Central High", Participant == "1":
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-14-1.svg)

]

---
# Our model  
.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | grouping structure), family = error distribution)
]
--
.br2.f4.green.bg-white[
&nbsp; &nbsp; &nbsp; lmer(ABS &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ~ age * interv + (1 + age &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; schoolid&nbsp; /&nbsp; ppt&nbsp; &nbsp; )
]

---
count: false
# Our model  
.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | grouping structure), family = error distribution)
]
.br2.f4.green.bg-white[
&nbsp; &nbsp; &nbsp; lmer(ABS &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ~ ageC * interv + (1 + ageC &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; schoolid&nbsp; /&nbsp; ppt&nbsp; &nbsp; )
]

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> schoolid </th>
   <th style="text-align:left;"> ppt </th>
   <th style="text-align:left;"> age </th>
   <th style="text-align:left;"> interv </th>
   <th style="text-align:left;"> ABS </th>
   <th style="text-align:left;"> ageC </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 12 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.5 </td>
   <td style="text-align:left;"> 0 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 13 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.4 </td>
   <td style="text-align:left;"> 1 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 14 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 53 </td>
   <td style="text-align:left;"> 2 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 15 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 70.7 </td>
   <td style="text-align:left;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
  </tr>
</tbody>
</table>

---
class: inverse, center, middle

<h2 style="text-align: left;opacity:0.3;">Part 1: Model Specification</h2>
<h2>Part 2: Model fitting</h2>
<h2 style="text-align: left;opacity:0.3;">Part 3: Interpretation & Inference</h2>
<h2 style="text-align: left;opacity:0.3;">Part 4: Reporting</h2>

---
# Model issues

- Check for convergence issues & singular fits.

```r
absmod <- lmer(ABS ~ ageC * interv + (1 + ageC | schoolid/ppt), data = absint,
     control = lmerControl(optimizer="bobyqa"), REML = TRUE)
```

```
## [1] "Normal exit from bobyqa"
```

- Check assumptions have been met

.pull-left[
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-19-1.svg)
]
.pull-right[
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-20-1.svg)
]

---
# Model issues

- Convergence issues.

```r
absmod <- lmer(ABS ~ ageC * interv + (1 + ageC | schoolid/ppt), data = absint,
     control = lmerControl(optimizer="bobyqa"), REML = TRUE)
```

```
## [1] "Normal exit from bobyqa"
```

- Check assumptions have been met

![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-23-1.svg)

---
# Model issues

- Convergence issues.

```r
absmod <- lmer(ABS ~ ageC * interv + (1 + ageC | schoolid/ppt), data = absint,
     control = lmerControl(optimizer="bobyqa"), REML = TRUE)
```

```
## [1] "Normal exit from bobyqa"
```

- Check assumptions have been met

- Do tests if you want, but beware. Multilevel data tends to have bigger `$n$`, and these tests are overly sensitive.  
Personally, I prefer plots.

```
## [1] "Shapiro-Wilk normality test resid(absmod) W=1, p=0.98"                          
## [2] "Shapiro-Wilk normality test ranef(absmod)$schoolid$`(Intercept)` W=0.97, p=0.69"
## [3] "Shapiro-Wilk normality test ranef(absmod)$schoolid$ageC W=0.95, p=0.34"         
## [4] "Shapiro-Wilk normality test ranef(absmod)$ppt$`(Intercept)` W=1, p=0.95"        
## [5] "Shapiro-Wilk normality test ranef(absmod)$ppt$ageC W=0.99, p=0.44"
```

---
# Model issues

- Convergence issues.

```r
absmod <- lmer(ABS ~ ageC * interv + (1 + ageC | schoolid/ppt), data = absint,
     control = lmerControl(optimizer="bobyqa"), REML = TRUE)
```

```
## [1] "Normal exit from bobyqa"
```

- Check assumptions have been met

![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-29-1.svg)

---
class: inverse, center, middle

<h2 style="text-align: left;opacity:0.3;">Part 1: Model Specification</h2>
<h2 style="text-align: left;opacity:0.3;">Part 2: Model fitting</h2>
<h2>Part 3: Interpretation & Inference</h2>
<h2 style="text-align: left;opacity:0.3;">Part 4: Reporting</h2>

---
# Fixed effects

.pull-left[

```r
fixef(absmod)
```

```
##  (Intercept)         ageC      interv1 ageC:interv1 
##       46.461        1.673        1.096       -1.352
```
]
.pull-right[
MAP IT TO THE PLOT!!!!
]

---
count: false
# Fixed effects

.pull-left[

```r
fixef(absmod)
```

```
##  (Intercept)         ageC      interv1 ageC:interv1 
##       46.461        1.673        1.096       -1.352
```
]
.pull-right[

```r
library(sjPlot)
plot_model(absmod, type="int")
```

![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-32-1.svg)
]

---
# Inference

.pull-left[
## Tests

- Model comparison

- Parameter estimates

]
.pull-right[
## Methods

- df approximations
  `parameters::model_parameterS(model, ci_method="kr")`
  `pbkrtest::KRmodcomp(model2,model1)`
- Likelihood Ratio Tests  
  `anova(model1, model2, ...)`
- Bootstrap
  - parametric bootstrap
    `pbkrtest::PBmodcomp()` and `confint(method="boot")`
  - case bootstrap
    `lmeresampler::bootstrap(model, type = "case", resample = c(....))`
]

---
# Inference

> **Research Questions**  
> 1. Does the presentation of aggressive behaviours increase as children enter the secondary school system? 
> 2. If so, is there any evidence for the effectiveness of Parent Management Training (PMT) group sessions in curbing the rise of aggressive behaviors during a child's transition into the secondary school system?

```r
fixef(absmod)
```

```
##  (Intercept)         ageC      interv1 ageC:interv1 
##       46.461        1.673        1.096       -1.352
```

---
# Inference

e.g.:

```r
parameters::model_parameters(absmod, ci_method="kr")
```

```
## # Fixed Effects
## 
## Parameter               | Coefficient |   SE |         95% CI |     t |    df |      p
## --------------------------------------------------------------------------------------
## (Intercept)             |       46.46 | 0.86 | [44.65, 48.27] | 53.72 | 18.53 | < .001
## ageC                    |        1.67 | 0.39 | [ 0.86,  2.49] |  4.33 | 17.98 | < .001
## interv [1]              |        1.10 | 1.21 | [-1.46,  3.65] |  0.90 | 17.39 | 0.378 
## ageC × interv [1] |       -1.35 | 0.55 | [-2.50, -0.20] | -2.48 | 17.26 | 0.024 
## 
## # Random Effects
## 
## Parameter                          | Coefficient
## ------------------------------------------------
## SD (Intercept: ppt:schoolid)       |        4.17
## SD (Intercept: schoolid)           |        1.58
## SD (ageC: ppt:schoolid)            |        2.22
## SD (ageC: schoolid)                |        0.67
## Cor (Intercept~ageC: ppt:schoolid) |        0.69
## Cor (Intercept~ageC: schoolid)     |        0.26
## SD (Residual)                      |        7.03
```

---
# Random effects

Use them to add context to results

- e.g. they can give a descriptive answer to "should we expect *all* children get more aggressive in secondary school?"

![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-36-1.svg)

---
# Random effects

Use them to add context to results

- e.g. they can give a descriptive answer to "should we expect *all* children get more aggressive in secondary school?"

![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-37-1.svg)

---
class: inverse, center, middle

<h2 style="text-align: left;opacity:0.3;">Part 1: Model Specification</h2>
<h2 style="text-align: left;opacity:0.3;">Part 2: Model fitting</h2>
<h2 style="text-align: left;opacity:0.3;">Part 3: Interpretation & Inference</h2>
<h2>Part 4: Reporting</h2>

---
# Reporting the analysis process

- Data cleaning outlier/data removal, transformations _prior to_ analysis.

- Unplanned transformations and data removal which are carried out in order to meet assumptions.

- Specify all fixed effects (explanatory variables & covariates).  
Link them to explicitly stated research questions/hypotheses.

- Explicitly state the hierarchical structure of the data and of the model.  
Specify random effects according to the sampling units (schools/children etc) with which they interact.

- Planned structure of random effects to be fitted

- Procedure to be used to decide on final random effect structure.

- State clearly the relevant test/comparison/parameter estimate of interest.  
Link to explicitly stated research questions/hypotheses.

- Method you plan to use to conduct inference (e.g. LRT, kr, bootstrap)
  - Any model comparisons should be clearly stated so that the reader understands the structure of *both* models being compared.

---
# Reporting results

.pull-left[
- Software packages and versions used to fit the model(s), along with the estimation method (ML/REML) and optimiser used.

- If proposed model failed to converge:  
  - steps leading to final converging model.  
  - final model structure

]

.pull-right[
For final model:

- all parameter estimates for fixed effects.

- coefficients
    - standard errors and/or confidence intervals
    - associated test statistics, df and p-values (if used)  
    
{{content}} 
]
 
--

- random effects

- standard deviation and/or variance for each random effect
  - correlations/covariances if modelled   
  - residual variance/standard deviation

---
# Reporting: text and tables

.pull-left[
Tables help a lot!

{{content}}
]

But *they are not a substitute for interpretation*

- Key parameters of interest should also be included in-text, with interpretation.

---
count: false
# Reporting: text and tables

.pull-left[
Tables help a lot!

But *they are not a substitute for interpretation*

- Key parameters of interest should also be included in-text, with interpretation.

```r
library(sjPlot)
tab_model(absmod, show.p = FALSE, df.method="kr")
```

]

.pull-right[
<small>
<table style="border-collapse:collapse; border:none;">
<tr>
<th style="border-top: double; text-align:center; font-style:normal; font-weight:bold; padding:0.2cm;  text-align:left; ">&nbsp;</th>
<th colspan="2" style="border-top: double; text-align:center; font-style:normal; font-weight:bold; padding:0.2cm; ">ABS</th>
</tr>
<tr>
<td style=" text-align:center; border-bottom:1px solid; font-style:italic; font-weight:normal;  text-align:left; ">Predictors</td>
<td style=" text-align:center; border-bottom:1px solid; font-style:italic; font-weight:normal;  ">Estimates</td>
<td style=" text-align:center; border-bottom:1px solid; font-style:italic; font-weight:normal;  ">CI</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; ">(Intercept)</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">46.46</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">44.65&nbsp;&ndash;&nbsp;48.27</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; ">ageC</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">1.67</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">0.86&nbsp;&ndash;&nbsp;2.49</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; ">interv [1]</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">1.10</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">&#45;1.46&nbsp;&ndash;&nbsp;3.65</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; ">ageC × interv [1]</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">&#45;1.35</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">&#45;2.50&nbsp;&ndash;&nbsp;-0.20</td>
</tr>
<tr>
<td colspan="3" style="font-weight:bold; text-align:left; padding-top:.8em;">Random Effects</td>
</tr>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&sigma;<sup>2</sup></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">49.47</td>
</tr>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&tau;<sub>00</sub> <sub>ppt:schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">17.38</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&tau;<sub>00</sub> <sub>schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">2.50</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&tau;<sub>11</sub> <sub>ppt:schoolid.ageC</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">4.91</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&tau;<sub>11</sub> <sub>schoolid.ageC</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">0.45</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&rho;<sub>01</sub> <sub>ppt:schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">0.69</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&rho;<sub>01</sub> <sub>schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">0.26</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">N <sub>schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">20</td>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm; border-top:1px solid;">Observations</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left; border-top:1px solid;" colspan="2">992</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">Marginal R<sup>2</sup> / Conditional R<sup>2</sup></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">0.034 / 0.650</td>
</tr>

</table>
</small>
]

---
# Visualising the model(s)

- Think about your questions

.pull-left[
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-40-1.svg)
]
.pull-right[
> 1. Does the presentation of aggressive behaviours increase as children enter the secondary school system? 
> 2. If so, is there any evidence for the effectiveness of Parent Management Training (PMT) group sessions in curbing the rise of aggressive behaviors during a child's transition into the secondary school system?
]

---
# Visualising the model(s)

.pull-left[
There's always `sjPlot::plot_model()`!
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-41-1.svg)
]
--
.pull-right[
But plotting manually gives you more control:
![](dapr3_2324_05b_rq2mod_files/figure-html/unnamed-chunk-42-1.svg)
]

---
class: inverse, center, middle, animated, rotateInDownLeft

# End
## Thanks for listening!