DAPR3 Lab Exercises
  • Linear mixed models
    • Identifying grouping structure
    • Modelling group-level variability using random effects
    • Building maximal models and interpreting LMM estimates
    • Troubleshooting, checking assumptions and diagnostics
    • LMMs: Practice write-up
  • Measurement & Dimension Reduction
    • Scale Scores & PCA
    • EFA
    • CFA
  • Week 11: Mock Exam SAQs

Week 11: Mock Exam SAQs

When and where?

  • The exam is scheduled to take place on the Thursday the 10th December at 09:30-11:30.
  • For most of you this will be in the Appleton Tower Concourse, but please check your personal outlook calendar.

What?

  • The exam will contain  15  14 multiple choice questions (MCQs) and  8  7 short answer questions (SAQs).
  • Each MCQ is 2 points, and the SAQs range from 3-14 marks. For the larger questions, we provide breakdowns of what points to cover in your answer (see examples below).

Week 4?

  • Due to the strikes impacting the DAPR3 course in Week 4, any questions covering content from Week 4 (1 MCQ and 1 SAQ) have been removed from the exam.
  • The result is that there a total of 90 marks available, and final grades will be calculated as a percentage of that total.

Examples

For example MCQs, please review the weekly quizzes from the course. On the Learn page, under Assessments, you can find a folder of “Revision Quizzes”. These are open versions of the weekly quizzes we have covered so far, that you can take as many times as you wish.

For example SAQs, the questions below provide some that are similar to those found in the main exam. We are publishing these early due to the early date of the exam, so please be aware that we may not have covered all of the content in these questions yet. Bulletpoints of key things to cover in an answer will be available after the Week 11 labs.


Example 1 - LMM interpretation [12 marks]

A company that makes “six-minute journals” is interested in finding out whether regular use of their product helps alleviate anger in people who struggle with anger management.

They recruited 200 people who were already enrolled in weekly anger management classes. They gave 100 of those people a journal. Each participant fileld out weekly assessments of anger levels for 10 weeks.

The company has the following data:

  • ppt: each participant’s unique ID
  • week: number representing week, ranging from 0 (first week with the journal) to 9 (tenth week with the journal).
  • journal: whether or not someone used a journal, either No (coded as 0) or Yes (coded as 1)

They fit a linear mixed model and got the following results:

 Linear mixed model fit by REML. t-tests use Satterthwaite's method [
 lmerModLmerTest]
 Formula: anger ~ week * journal + (1 + week | ppt)
    Data: df
 
 REML criterion at convergence: 12665
 
 Scaled residuals: 
    Min     1Q Median     3Q    Max 
 -4.191 -0.649  0.019  0.619  3.397 
 
 Random effects:
  Groups   Name        Variance Std.Dev. Corr 
  ppt      (Intercept)  2.21    1.49          
           week         1.13    1.06     0.11 
  Residual             24.52    4.95          
 Number of obs: 2000, groups:  ppt, 200
 
 Fixed effects:
                 Estimate Std. Error      df t value Pr(>|t|)    
 (Intercept)        9.764      0.327 197.991   29.88   <2e-16 ***
 week              -0.141      0.119 198.003   -1.18   0.2402    
 journalYes         0.074      0.462 197.991    0.16   0.8730    
 week:journalYes   -0.467      0.169 198.003   -2.77   0.0062 ** 
 ---
 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1


  1. Interpret each of the fixed effects, rounding all numbers to two decimal places. You do not need to comment on the statistical significance of each coefficient. [4 marks]

  2. Give the range in which 95% of non-journalling participants’ intercepts are estimated to fall [2 marks], and the range in which 95% of non-journalling participants’ slopes are estimated to fall [2 marks]. Round the bounds of this range to two decimal places.

  3. The correlation between random intercepts and random slopes is 0.11 What does that correlation mean in the context of this study? [1 mark]

  4. For each of the following statements, write “Yes” if it’s a valid conclusion you can draw from the model, and write “No” if it is not.

  1. People who did journal had increasingly higher anger as the weeks went on. [1 mark]
  2. In both journalling conditions, anger decreased significantly as the weeks went on. [1 mark]
  3. Journalling is associated with a significantly greater reduction in anger over time, compared to just taking the anger management classes. [1 mark]

Solution 1. (a) max 4 marks

Intercept: The estimated average anger score for non-journalling participants in the first week of the study is 9.76 points. [1 mark]

week: For non-journalling participants, moving forward one week in the study is associated with a decrease in anger score of 0.14 points on average. [1 mark]

journalYes: In the first week of the study, participants in the journalling condition are estimated to have anger scores 0.07 points higher, on average, than participants in the non-journalling condition. [1 mark]

week:journalYes: For journalling participants, the association between the week of the study and anger score decreases by a further 0.47 points. [1 mark] (In other words, the simple slope over week for journalling participants is -0.14 + -0.47 = -0.61; calculating this simple slope would also be worth 1 mark.)


(b) max 4 marks

Estimated range of 95% of by-subject intercepts for non-journallers (that is, for the reference level of journal)

  • Lower bound [1 mark]: 9.76 – 2 * 1.49 = 6.78
  • Upper bound [1 mark]: 9.76 + 2 * 1.49 = 12.74

Estimated range of 95% of by-subject adjustments to slope over week for non-journallers:

  • Lower bound [1 mark]: -0.14 – 2 * 1.06 = -2.26
  • Upper bound [1 mark]: -0.14 + 2 * 1.06 = 1.98


(c) max 1 mark

Something like:

  • Participants with higher-than-average anger scores in the first week of the study tend (very slightly—it’s a fairly weak correlation) to show less improvement in their anger scores as the weeks go on.


(d) max 3 marks

  1. No
  2. No
  3. Yes

Example 2 - LMM [4 marks]

A categorical variable such as group_var can be included in LMM as either:
A lmer(outcome ~ ....... + (1 | group_var) ....
or
B lmer(outcome ~ ....... + group_var + ....

How do these differ? You may want to comment on, for example:

  • the type of variability that we think group_var contributes
  • what inferential information you get from the model (that is, what hypotheses can you test)
  • what groups the model results generalise to

Solution 2.

  • B treats it as a fixed effect, A as a random effect
  • B estimates differences between each group as contributing reproducible variation in the outcome
  • A estimates the variance between groups as a source of variation in the outcome that is due to random sampling
  • A generalises to the broader population from which the groups are a sample of, B generalises to the specific groups used
  • A can be easily extended to let slopes vary by groups
  • B if we are interested in the differences between groups, A if we just want to say “groups vary”
  • B gives us SEs, p-values etc for group differences, A doesn’t

Example 3 - EFA [14 marks]

Below are the results of an analysis conducted on a set of 12 items assessing environmental conscientiousness. Participants are asked to respond to each statement on a 5-point likert scale from “Strongly Disagree” to “Strongly Agree”.

Based on the results and the item descriptions below, provide an interpretation of the factor solution. Your description should include:

  1. Briefly describe the analysis that has been conducted. [2 marks]
  2. Comment on the numerical solution of the fitted model. [5 marks]
  3. Is the selected number of factors suitable, or is it under/over extracted? Why or why not / how can you tell? [2 marks]
  4. What is a reasonable next step in this analysis? [1 mark] Explain your reasoning. [1 mark]
  5. Do your best to label/define each factor. [3 marks]

item wording
y1 I recycle regularly
y2 I use eco-friendly transportation
y3 I buy sustainable products
y4 I know how to reduce my carbon footprint
y5 Protecting resources matters to me
y6 I care about protecting the environment
item wording
y7 I feel responsible for my environmental impact
y8 I am worried about climate change effects
y9 I know about the harm of single-use plastics
y10 I know how deforestation affects climate change
y11 I know about relevant environmental policies
y12 Wildlife destruction concerns me deeply

Solution 3.

Important
This is not a “rubric”, in the sense that a) it is not an exhaustive list of possible things that could be mentioned and b) the bullet points below are not directly mapped to “marks” (i.e., 1 bullet point does not always equal 1 mark)

  1. Briefly describe the analysis that has been conducted. [2 marks]

    • 12 items, 4 factors extracted
    • oblique rotation allowing factors to correlate
    • estimated with maximum likelihood
  2. Comment on the numerical solution of the fitted model. [5 marks]

    • V explains 29% variance
    • I Factors ML2 & ML1 both have \(\geq 3\) salient/primary loadings
      • (salient = \(\geq |0.3|\))
      • Factors ML3 & ML4 do not
    • S some items (y5,y6) have no salient loadings
    • C 3 complex items (y4, y5, y6)
    • H no heywood cases
    • Factors ML1,2,3 correlated weak-moderate, ML4 not strongly correlated with others. Suggests distinct consructs.
  3. Is the selected number of factors suitable, or is it under/over extracted? Why or why not / how can you tell? [2 marks]

    • probably overextracted (too many factors)
    • because: lack of clarity/definition of ML3 & 4
      • complex items y5 & y6 are spread across ML3 & ML4 - a 3 factor solution may well make more sense
      • both ML3 & 4 have only 1 item with salient loading
      • 3rd factor explaining only 6%, 4th factor only 2%
      • SSloadings for ML3 & ML4 are all <1
  4. What is a reasonable next step in this analysis? [1 mark] Explain your reasoning. [1 mark]

    • examine 3 factor solution?
    • consider removing item 4 (wording makes it sit across ML1 & 2, combining “knowledge” & “behaviours”)?
    • once satisfactory solution obtained, collect data on new sample and test if model replicates
  5. Do your best to label/define each factor. [3 marks]

    • ML1 = “environmental knowledge”
    • ML2 = “environmental behaviours”
    • ML3/4 not well defined enough to name
    • combined ML3/4 look like might become “environmental concern”.

Example 4 - SSloadings [6 marks]

Calculate the 6 values missing from the table below: SSloadings [2 marks] and proportion variance [2 marks] & cumulative variance [2 marks].

       PC1   PC2
item1 0.90  0.00
item2 0.90 -0.29
item3 0.90  0.30
item4 0.70  0.70
item5 0.81 -0.50
               PC1  PC2 
SS loadings     __   __ 
Proportion Var  __   __ 
Cumulative Var  __   __ 

Solution 4. Table filled in:

                 PC1   PC2
SS loadings    3.576 0.914
Proportion Var 0.715 0.183
Cumulative Var 0.715 0.898

How?
start by squaring all the numbers, and sum the columns to give us SSloadings:

        PC1    PC2
item1 0.810 0.0000
item2 0.810 0.0841
item3 0.810 0.0900
item4 0.490 0.4900
item5 0.656 0.2500
Sum   3.576 0.9141

divide SSloadings by 5 (because 5 observed variables) to get proportion variance

  PC1   PC2 
0.715 0.183 

those two numbers are then cumulatively summed for cumulative variance:

  PC1   PC2 
0.715 0.898