Multi-level Model Research Questions

class: center, middle, inverse, title-slide

.title[
# <b>Multi-level Model Research Questions</b>
]
.subtitle[
## Data Analysis for Psychology in R 3
]
.author[
### Josiah King
]
.institute[
### Department of Psychology<br/>The University of Edinburgh
]

---

class: inverse, center, middle

<h2>Part 1: Model Specification</h2>
<h2 style="text-align: left;opacity:0.3;">Part 2: Analysis</h2>
<h2 style="text-align: left;opacity:0.3;">Part 3: Reporting</h2>

---
class: center, middle
# How do we get from research questions and study design to model specification?

---
# Study 
In 2010 A US state's commissioner for education approved the trialing of an intervention in which yearly Parent Management Training (PMT) group sessions were offered to the parents of a cohort of students entering 10 different high schools. Every year, the parents were asked to fill out an informant-based version of the Aggressive Behaviour Scale (ABS), measuring verbal and physical abuse, socially inappropriate behavior, and resisting care. Where possible, the same parents were followed up throughout the child's progression through high school.  
Alongside this, parents from a cohort of students entering 10 further high schools in the state were recruited to also complete the same informant-based ABS, but were not offered the PMT group sessions.

1. How does the presentation of aggressive behaviours change with age?

2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression?

The data is available at https://uoepsy.github.io/data/abs_intervention.csv

---
# Data

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> variable </th>
   <th style="text-align:left;"> description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> schoolid </td>
   <td style="text-align:left;"> Name of school </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ppt </td>
   <td style="text-align:left;"> Participant number </td>
  </tr>
  <tr>
   <td style="text-align:left;"> age </td>
   <td style="text-align:left;"> Age of participant (in years) at observation </td>
  </tr>
  <tr>
   <td style="text-align:left;"> interv </td>
   <td style="text-align:left;"> Informant-based Aggressive Behaviour Scale (ABS) score (range 0 to 100) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ABS </td>
   <td style="text-align:left;"> Whether or not the school was part of the intervention group in which Parent Management Training (PMT) group sessions were offered to the parents </td>
  </tr>
</tbody>
</table>

```
## # A tibble: 6 x 5
##   schoolid                 ppt   age interv   ABS
##   <chr>                  <dbl> <dbl> <fct>  <dbl>
## 1 Blue River High School     1    12 1       47.5
## 2 Blue River High School     1    13 1       47.4
## 3 Blue River High School     1    14 1       53.0
## 4 Blue River High School     1    15 1       70.7
## 5 Blue River High School     1    16 1       66.3
## 6 Blue River High School     1    17 1       NA
```

---
# Outcome and Fixed Effects

.br2.f4.gray.bg-white[
(g)lmer(**outcome** ~ fixed effects + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- What are we interested in explaining/predicting?

]

.pull-right[
> **Research Questions**
> 1. How does the presentation of aggressive behaviours change with age?

>2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression?

]

---
count:false
# Outcome and Fixed Effects

.br2.f4.gray.bg-white[
**(g)lmer**(outcome ~ fixed effects + (random effects | grouping structure), **family = error distribution**)
]

.pull-left[

- What are we interested in explaining/predicting?

- How is this measured?

]
.pull-right[
> **Research Questions**
> 1. How does the presentation of aggressive behaviours change with age?

>2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression?

![](05_mlmresearch_files/figure-html/unnamed-chunk-10-1.svg)

]

---
count:false
# Outcome and Fixed Effects

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ **fixed effects** + (random effects | grouping structure), family = error distribution)
]

.pull-left[

- What are we interested in explaining/predicting?

- How is this measured?

- What variables are we interested in explaining this by?

]
.pull-right[
> **Research Questions**
> 1. How does the presentation of aggressive behaviours change with age?

>2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression?

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> schoolid </th>
   <th style="text-align:left;"> ppt </th>
   <th style="text-align:left;"> age </th>
   <th style="text-align:left;"> interv </th>
   <th style="text-align:left;"> ABS </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 12 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.5 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 13 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 14 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 53 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 15 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 70.7 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
  </tr>
</tbody>
</table>

]

---
# Within & Between Effects

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ **fixed effects** + (random effects | grouping structure), family = error distribution)
]

.pull-left[

- Are our questions about the effects of our predictors specifically in reference to _group means_ of predictors?

- "the effect of being higher on `$x$` *__for a__* group"  
  
  - "the effect of a group being *__on average__* higher on `$x$`"

]
.pull-right[
> **Research Questions**
> 1. How does the presentation of aggressive behaviours change with age?

>2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression?

]

---
# The Grouping Structure

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

]
--
.pull-right[
<table>
 <thead>
  <tr>
   <th style="text-align:left;"> schoolid </th>
   <th style="text-align:left;"> ppt </th>
   <th style="text-align:left;"> age </th>
   <th style="text-align:left;"> interv </th>
   <th style="text-align:left;"> ABS </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 12 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.5 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 13 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 47.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 14 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 53 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Blue River High School </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 15 </td>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> 70.7 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
   <td style="text-align:left;"> ... </td>
  </tr>
</tbody>
</table>
]

---
# The Grouping Structure

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- Of the different ways we can group our data, which groupings are of specific inferential interest?

- Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?

]
.pull-right[
> **Research Questions**
> 1. How does the presentation of aggressive behaviours change with age?

>2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression?

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ **fixed effects** + (random effects | grouping structure), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- **Of the different ways we can group our data, which groupings are of specific inferential interest?**

- Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?

]
.pull-right[
> **Research Questions**
> 1. How does the presentation of aggressive behaviours change with age?

>2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression?

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | **grouping structure**), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- Of the different ways we can group our data, which groupings are of specific inferential interest?

- **Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?**

]
.pull-right[
> **Research Questions**
> 1. How does the presentation of aggressive behaviours change with age?

>2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression?

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | **grouping structure**), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- Of the different ways we can group our data, which groupings are of specific inferential interest?

- Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?

- Is there more than one grouping of this sort, and if so, are these groupings nested? Are the labels unique?  
  
  - For each level, how many groups have we sampled?  
]
--
.pull-right[

```
## [1] 1568
```

```
## # A tibble: 1 x 6
##   schoolid   ppt   age interv   ABS schoolppt
##      <int> <int> <int>  <int> <int>     <int>
## 1       20    16     8      2   993       196
```

]
---
count: false
# The Grouping Structure

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | **grouping structure**), family = error distribution)
]

.pull-left[
- In what different ways can we group our data?

- Of the different ways we can group our data, which groupings are of specific inferential interest?

- Of the different ways we can group our data, which groupings do we think of as a random sample from a general population of groups?

- Is there more than one grouping of this sort, and if so, are these groupings nested? Are the labels unique?  
  
  - For each level, how many groups have we sampled?  
]
.pull-right[

```
##                                  
##                                   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
##   Blue River High School          8 8 8 8 8 8 8 8 8  8  0  0  0  0  0  0
##   Central Grammar School          8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Central High                    8 8 8 8 8 8 8 8 8  0  0  0  0  0  0  0
##   Clearwater Charter School       8 8 8 8 8 8 8 8 8  8  8  8  8  8  8  0
##   Clearwater School of Fine Arts  8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Coral Coast Institute           8 8 8 8 8 8 8 8 8  0  0  0  0  0  0  0
##   Eagle Mountain Technical School 8 8 8 8 8 8 8 0 0  0  0  0  0  0  0  0
##   Eastwood Institute              8 8 8 8 8 8 8 8 8  8  0  0  0  0  0  0
##   Edgewood Grammar School         8 8 8 8 8 8 8 8 8  8  0  0  0  0  0  0
##   Elk Grove School                8 8 8 8 8 8 8 8 0  0  0  0  0  0  0  0
##   Grand Mountain Elementary       8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Horizon Elementary              8 8 8 8 8 0 0 0 0  0  0  0  0  0  0  0
##   Liberty High                    8 8 8 8 8 8 8 8 0  0  0  0  0  0  0  0
##   Long Beach School               8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Oak Park Technical School       8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Seal Bay Middle School          8 8 8 8 8 8 8 8 8  0  0  0  0  0  0  0
##   South Fork Conservatory         8 8 8 8 8 8 8 8 0  0  0  0  0  0  0  0
##   Tranquillity Elementary         8 8 8 8 8 8 0 0 0  0  0  0  0  0  0  0
##   Tranquillity Middle School      8 8 8 8 8 8 8 8 8  8  8  0  0  0  0  0
##   Woodside University             8 8 8 8 8 8 8 8 8  8  8  8  8  8  8  8
```

]
---
# Random Intercepts and Slopes

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (**random effects** | grouping structure), family = error distribution)
]

.pull-left[

- Which of our fixed effects can vary for our random groups?

- "does a single group have multiple _distinct_ values for `$x$`?"  
  
  - "for the data from only one of our groups, can we estimate `$y \sim x$`?"

]
--
.pull-right[
All the data:
![](05_mlmresearch_files/figure-html/unnamed-chunk-18-1.svg)
Data from School == "Central High":
![](05_mlmresearch_files/figure-html/unnamed-chunk-19-1.svg)

]

---
# Random Intercepts and Slopes

.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (**random effects** | grouping structure), family = error distribution)
]

.pull-left[

- Which of our fixed effects can vary for our random groups?

- "does a single group have multiple values for `$x$`?"  
  
  - "for the data from only one of our groups, can we estimate `$y \sim x$`?"

]
.pull-right[
All the data:
![](05_mlmresearch_files/figure-html/unnamed-chunk-20-1.svg)
Data from School == "Central High", Participant == "1":
![](05_mlmresearch_files/figure-html/unnamed-chunk-21-1.svg)

]

---
# Our model  
.br2.f4.gray.bg-white[
(g)lmer(outcome ~ fixed effects + (random effects | grouping structure), family = error distribution)
]
--
.br2.f4.green.bg-white[
&nbsp; &nbsp; &nbsp; lmer(ABS &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ~ age * interv + (1 + age &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | &nbsp; &nbsp; schoolid&nbsp; /&nbsp; ppt&nbsp; &nbsp; )
]

---
class: inverse, center, middle

<h2 style="text-align: left;opacity:0.3;">Part 1: Model Specification</h2>
<h2>Part 2: Analysis</h2>
<h2 style="text-align: left;opacity:0.3;">Part 3: Reporting</h2>

---
# Model issues

- Check for convergence issues & singular fits.

```
## [1] "Normal exit from bobyqa"
```

- Check assumptions have been met

![](05_mlmresearch_files/figure-html/unnamed-chunk-26-1.svg)

---
# Model issues

- Convergence issues.

```
## [1] "Normal exit from bobyqa"
```

- Check assumptions have been met

.pull-left[
![](05_mlmresearch_files/figure-html/unnamed-chunk-29-1.svg)
]
.pull-right[
![](05_mlmresearch_files/figure-html/unnamed-chunk-30-1.svg)
]

---
# Model issues

- Convergence issues.

```
## [1] "Normal exit from bobyqa"
```

- Check assumptions have been met

```
## [1] "Shapiro-Wilk normality test resid(absmod) W=1, p=0.98"                          
## [2] "Shapiro-Wilk normality test ranef(absmod)$schoolid$`(Intercept)` W=0.97, p=0.69"
## [3] "Shapiro-Wilk normality test ranef(absmod)$schoolid$age W=0.95, p=0.34"          
## [4] "Shapiro-Wilk normality test ranef(absmod)$ppt$`(Intercept)` W=1, p=0.95"        
## [5] "Shapiro-Wilk normality test ranef(absmod)$ppt$age W=0.99, p=0.44"
```

---
# Model issues

- Convergence issues.

```
## [1] "Normal exit from bobyqa"
```

- Check assumptions have been met

![](05_mlmresearch_files/figure-html/unnamed-chunk-36-1.svg)

---
# Inference

.pull-left[
## Tests

- Model comparison

- Parameter estimates

]
.pull-right[
## Methods

- df approximations
  
- Likelihood Ratio Tests  
  `anova(model1, model2, ...)`
- Bootstrap
  - parametric bootstrap
    `pbkrtest::PBmodcomp()` and `confint(method="boot")`
  - case bootstrap
    `lmeresampler::bootstrap(model, type = "case", resample = c(....))`
]

---
# Inference

e.g.:

```
Random effects:
 Groups       Name        Variance Std.Dev. Corr
 ppt:schoolid (Intercept) 17.379   4.169        
              age          4.913   2.217    0.69
 schoolid     (Intercept)  2.505   1.583        
              age          0.448   0.669    0.26
 Residual                 49.465   7.033        
Number of obs: 992, groups:  ppt:schoolid, 196; schoolid, 20

Fixed effects:
            Estimate Std. Error     df t value Pr(>|t|)    
(Intercept)   46.461      0.862 17.915   53.88   <2e-16 ***
age            1.673      0.385 20.361    4.34   0.0003 ***
interv1        1.096      1.207 16.818    0.91   0.3766    
age:interv1   -1.352      0.543 19.560   -2.49   0.0218 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
```

---
class: inverse, center, middle

<h2 style="text-align: left;opacity:0.3;">Part 1: Model Specification</h2>
<h2 style="text-align: left;opacity:0.3;">Part 2: Analysis</h2>
<h2>Part 3: Reporting</h2>

---
# Reporting the analysis process

- Data cleaning outlier/data removal, transformations _prior to_ analysis.

- Unplanned transformations and data removal which are carried out in order to meet assumptions.

- Specify all fixed effects (explanatory variables & covariates).  
Link them to explicitly stated research questions/hypotheses.

- Explicitly state the hierarchical structure of the data and of the model.  
Specify random effects according to the sampling units (schools/children etc) with which they interact.

- State the software packages and versions used to fit the model(s), along with the estimation method (ML/REML) and optimiser used.

- If a proposed model fails to converge, clearly specify the procedure used to obtain converging model.

- State clearly the relevant test/comparison/parameter estimate of interest.  
Link to explicitly stated research questions/hypotheses.  
  - Any model comparisons should be clearly stated so that the reader understands the structure of both models being compared.

- Specify the method you plan to use to conduct inference (e.g. LRT, bootstrap)

---
# Reporting results from the model(s)

.pull-left[
Information to include:

- all parameter estimates for fixed effects.

- coefficients
    - standard errors and/or confidence intervals
    - associated test statistics and p-values (if used)  
    
{{content}} 
]  
--

- random effects

- standard deviation and/or variance for each random effect
  - correlations/covariances if modelled   
  - residual variance/standard deviation
  
{{content}}
  
--

- some measure of model fit (marginal/conditional `$R^2$`)

---
count:false
# Reporting results from the model(s)

.pull-left[
Information to include:

- all parameter estimates for fixed effects.

- coefficients
    - standard errors and/or confidence intervals
    - associated test statistics and p-values (if used)  
    
- random effects

- standard deviation and/or variance for each random effect
  - correlations/covariances if modelled  
  - residual variance/standard deviation  
  
- some measure of model fit (marginal/conditional `$R^2$`)

<br>
`sjPlot::tab_model()` gets you a lot of the way there!

]
.pull-right[
<small>
<table style="border-collapse:collapse; border:none;">
<tr>
<th style="border-top: double; text-align:center; font-style:normal; font-weight:bold; padding:0.2cm;  text-align:left; ">&nbsp;</th>
<th colspan="2" style="border-top: double; text-align:center; font-style:normal; font-weight:bold; padding:0.2cm; ">ABS</th>
</tr>
<tr>
<td style=" text-align:center; border-bottom:1px solid; font-style:italic; font-weight:normal;  text-align:left; ">Predictors</td>
<td style=" text-align:center; border-bottom:1px solid; font-style:italic; font-weight:normal;  ">Estimates</td>
<td style=" text-align:center; border-bottom:1px solid; font-style:italic; font-weight:normal;  ">CI</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; ">(Intercept)</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">46.46</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">44.77&nbsp;&ndash;&nbsp;48.15</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; ">age</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">1.67</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">0.92&nbsp;&ndash;&nbsp;2.43</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; ">interv [1]</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">1.10</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">&#45;1.27&nbsp;&ndash;&nbsp;3.47</td>
</tr>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; ">age * interv [1]</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">&#45;1.35</td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:center;  ">&#45;2.42&nbsp;&ndash;&nbsp;-0.29</td>
</tr>
<tr>
<td colspan="3" style="font-weight:bold; text-align:left; padding-top:.8em;">Random Effects</td>
</tr>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&sigma;<sup>2</sup></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">49.47</td>
</tr>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&tau;<sub>00</sub> <sub>ppt:schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">17.38</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&tau;<sub>00</sub> <sub>schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">2.50</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&tau;<sub>11</sub> <sub>ppt:schoolid.age</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">4.91</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&tau;<sub>11</sub> <sub>schoolid.age</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">0.45</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&rho;<sub>01</sub> <sub>ppt:schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">0.69</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">&rho;<sub>01</sub> <sub>schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">0.26</td>

<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm;">N <sub>schoolid</sub></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left;" colspan="2">20</td>
<tr>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; text-align:left; padding-top:0.1cm; padding-bottom:0.1cm; border-top:1px solid;">Marginal R<sup>2</sup> / Conditional R<sup>2</sup></td>
<td style=" padding:0.2cm; text-align:left; vertical-align:top; padding-top:0.1cm; padding-bottom:0.1cm; text-align:left; border-top:1px solid;" colspan="2">0.034 / 0.650</td>
</tr>

</table>
</small>
]

---
# Visualising the model(s)

- Think about your questions

.pull-left[
![](05_mlmresearch_files/figure-html/unnamed-chunk-40-1.svg)
]
.pull-right[
> **Research Questions**
> 1. How does the presentation of aggressive behaviours change with age?

>2. Is there any evidence for the efficacy of Parent Management Training (PMT) group sessions in reducing levels of adolescent aggression? 
]

---
# Visualising the model(s)

.pull-left[
There's always `sjPlot::plot_model()`!
![](05_mlmresearch_files/figure-html/unnamed-chunk-41-1.svg)
]
--
.pull-right[
But plotting manually gives you more control:
![](05_mlmresearch_files/figure-html/unnamed-chunk-42-1.svg)
]

---
class: inverse, center, middle, animated, rotateInDownLeft

# End
## Thanks for listening!