Writing-up

LEARNING OBJECTIVES

1. Construct a linear model from information in published paper.
2. Evaluate a published analysis with respect to the thoroughness of information presented.
3. Write-up and provide interpretation of a linear model analysis with multiple predictors

Write-up task

Here, we’re going to walk through a high-level step-by-step guide of what to include in a write-up of a statistical analysis. We’re going to use an example analysis using one of the datasets we have worked with on a number of exercises in previous labs concerning personality traits, social comparison, and depression and anxiety.

The aim in writing should be that a reader is able to more or less replicate your analyses without referring to your R code. This requires detailing all of the steps you took in conducting the analysis.
The point of using RMarkdown is that you can pull your results directly from the code. If your analysis changes, so does your report!

You can find a .pdf of the take-everywhere write-up checklist here.

Research question and analysis. Click the plus to expand →

Research question
Previous research has identified an association between an individual’s perception of their social rank and symptoms of depression, anxiety and stress. We are interested in the individual differences in this relationship.
Specifically:

• Controlling for other personality traits, does neuroticism moderate effects of social comparison on symptoms of depression, anxiety and stress?

library(tidyverse) # for all things!
library(psych) # good for descriptive stats
library(car) # for assumption tests
library(sjPlot) # for plotting models

scs_study <- read_csv("https://uoepsy.github.io/data/scs_study.csv")

# scale scs score
scs_study <- 
  scs_study %>% 
    mutate(
      zscs = (scs-mean(scs))/sd(scs)
    )

# the describe() function is from the psych package
describe(scs_study)

##      vars   n  mean   sd median trimmed  mad   min   max range  skew kurtosis
## zo      1 656  0.09 1.02   0.08    0.08 1.07 -2.82  3.55  6.37  0.15    -0.08
## zc      2 656  0.02 1.00   0.00    0.03 1.03 -3.22  3.08  6.30 -0.08    -0.13
## ze      3 656  0.00 1.00  -0.04   -0.01 1.00 -3.01  2.80  5.81  0.05    -0.13
## za      4 656  0.00 1.00  -0.02    0.01 1.04 -2.94  2.97  5.91 -0.08    -0.04
## zn      5 656  0.00 1.00  -0.21   -0.10 1.00 -1.45  3.35  4.80  0.80     0.04
## scs     6 656 35.77 3.53  35.00   35.59 2.97 27.00 54.00 27.00  0.60     0.96
## dass    7 656 44.72 6.76  44.00   44.62 5.93 23.00 68.00 45.00  0.18     0.33
## zscs    8 656  0.00 1.00  -0.22   -0.05 0.84 -2.48  5.16  7.64  0.60     0.96
##        se
## zo   0.04
## zc   0.04
## ze   0.04
## za   0.04
## zn   0.04
## scs  0.14
## dass 0.26
## zscs 0.04

dass_mdl <- lm(dass ~ 1 + zscs*zn + zo + zc + ze + za, data = scs_study)
plot(dass_mdl)

##       35 
## 2.664798

dass_mdl2 <- lm(dass ~ 1 + zscs*zn + zo + zc + ze + za, data = scs_study[-35, ])

# linearity
plot(dass_mdl2, which=1)

# equal variances
residualPlots(dass_mdl2)

##            Test stat Pr(>|Test stat|)  
## zscs          1.8141          0.07013 .
## zn           -0.5911          0.55467  
## zo            1.7801          0.07553 .
## zc           -0.2403          0.81018  
## ze           -0.9951          0.32004  
## za            0.0725          0.94219  
## Tukey test   -1.6406          0.10089  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

ncvTest(dass_mdl2)

## Non-constant Variance Score Test 
## Variance formula: ~ fitted.values 
## Chisquare = 1.508659, Df = 1, p = 0.21934

# normality
shapiro.test(residuals(dass_mdl2))

## 
##  Shapiro-Wilk normality test
## 
## data:  residuals(dass_mdl2)
## W = 0.99826, p-value = 0.7616

# independence
dwt(dass_mdl2)

##  lag Autocorrelation D-W Statistic p-value
##    1     -0.03991314      2.078413   0.326
##  Alternative hypothesis: rho != 0

# multicollinearity
vif(dass_mdl2)

##     zscs       zn       zo       zc       ze       za  zscs:zn 
## 1.015133 1.015736 1.013310 1.008235 2.332486 2.342220 1.012475

summary(dass_mdl2)

## 
## Call:
## lm(formula = dass ~ 1 + zscs * zn + zo + zc + ze + za, data = scs_study[-35, 
##     ])
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -17.1455  -3.8155  -0.0066   3.6905  18.1483 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 44.97703    0.22635 198.708  < 2e-16 ***
## zscs        -1.93818    0.23042  -8.412 2.58e-16 ***
## zn           1.41639    0.22661   6.250 7.44e-10 ***
## zo          -0.31435    0.22056  -1.425    0.155    
## zc           0.09134    0.22515   0.406    0.685    
## ze           0.52695    0.34233   1.539    0.124    
## za           0.33847    0.34281   0.987    0.324    
## zscs:zn     -2.76609    0.24097 -11.479  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 5.733 on 647 degrees of freedom
## Multiple R-squared:  0.279,  Adjusted R-squared:  0.2712 
## F-statistic: 35.76 on 7 and 647 DF,  p-value: < 2.2e-16

Think

What do you know? What do you hope to learn? What did you learn during the exploratory analysis?

1: Describe design

If you were reporting on your own study, then the first you would want to describe the study design, the data collection strategy, etc.
This is not necessary here, but we could always say something brief like:

Data was obtained from https://uoepsy.github.io/data/scs_study.csv: a dataset containing information on 656 participants

2: Describe the data

• How many observational units?
  
• Are there any observations that have been excluded based on pre-defined criteria? How/why, and how many?
• Describe and visualise the variables of interest. How are they scored? have they been transformed at all?
• Describe and visualise relationships between variables. Report covariances/correlations.

Solution

Data was obtained from https://uoepsy.github.io/data/scs_study.csv: a dataset containing information on 656 participants, including Z-scores on the 5 personality traits assessed by the Big-Five Aspects Scale (BFAS) (Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism). Participants were also assessed on the Social Comparison Scale (SCS), which is an 11-item scale measuring self-perception (relative to others) of social rank, attractiveness and belonging, and the Depression Anxiety and Stress Scale (DASS-21) - a 21 item measure with higher scores indicating higher severity of symptoms. For both of these measures, only total scores are available. Items in the SCS are measured on a 5-point scale, giving minimum and maximum possible scores of 11 and 55 respectively. Items in the DASS-21 are measured on a 4-point scale, meaning that scores can range from a possible 21 to 84.

All participant data was complete (no missing values), with scores on the SCS and the DASS-21 all within possible ranges (see Table 1). Bivariate correlations show a moderate negative relationship between DASS-21 and SCS scores; a moderate positive relationship between DASS-21 and Neuroticism, and a weak positive correlation between SCS and Neuroticism. Additionally, a strong positive relationship is evident between Extraversion and Agreeableness (see Figure 1).

Table 1: SCS and DASS-21 descriptive statistics

	n	mean	sd	min	max
scs	656	35.77	3.53	27	54
dass	656	44.72	6.76	23	68

Figure 1: Bivariate scatter plots (below diagonal), histograms (diagonal), and Pearson correlation coefficient (above diagonal), of personality trait measures and scores on the SCS and the DASS-21

3: Describe the analytical approach

• What type of statistical analysis do you use to answer the research question? (e.g., t-test, simple linear regression, multiple linear regression)
• Describe the model/analysis structure
• What is your outcome variable? What is its type?
• What are your predictors? What are their types?
• Any other specifics?

Solution

To investigate whether, when controlling for other personality traits, neuroticism moderates the effect of social comparison on symptoms of depression, anxiety and stress, total scores on the DASS-21 were modelled using multiple linear regression. The Z-scored measures on each of the big-five personality traits were included as predictors, along with scores on the SCS (Z-scored) and its interaction with the measure of Neuroticism. Effects will be considered statistically significant at \(\alpha = 0.01\).

4: Planned analysis vs actual analysis

• Was there anything you had to do differently than planned during the analysis? Did the modelling highlight issues in your data?
• Did you have to do anything (e.g., transform any variables, exclude any observations) in order to meet assumptions?

Solution

One observation was excluded from the final analysis as it was judged to be too influential on the model (Cook’s Distance = 2.66).

Show

Show the mechanics and visualisations which will support your conclusions

5: Present and describe final model

Present and describe the model or test which you deemed best to answer your question.

Solution

The final model was fitted to the remaining 655 observations, and took the form: \[ \text{DASS-21} = \beta_0 + \beta_1 \text{O} + \beta_2 \text{C} + \beta_3 \text{E} + \beta_4 \text{A} + \beta_5 \text{N} + \beta_6 \text{SCS} + \beta_7 \text{SCS} \cdot \text{N} + \epsilon \\ \begin{align} \text{Where} \\ & \text{O = Openness} \\ & \text{C = Conscientiousness} \\ & \text{E = Extraversion} \\ & \text{A = Agreeableness} \\ & \text{N = Neuroticism} \\ \end{align} \]

To address the research question of whether neuroticism moderates the effect of social comparison on depression and anxiety, we will consider the hypothesis test that the interaction coefficient is equal to zero, where:

\(H_0: \beta_7 = 0\). The interaction between SCS and Neuroticism is equal to zero.
\(H_1: \beta_7 \neq 0\). The interaction between SCS and Neuroticism is not equal to zero.

6: Are the assumptions and conditions of your final test or model satisfied?

For the final model (the one you report results from), were all assumptions met? (Hopefully yes, or there is more work to do…). Include evidence (tests or plots).

Solution

The model model met assumptions of linearity (see plot of model residuals vs fitted values, Figure 2), homoscedasticity (non-constant variance test indicated no evidence against the null hypothesis that the error variance is constant across level of the response, \(\chi^2(1)\)=1.51, \(p\)=0.219), independence of errors (Durbin-Watson test for autocorrelation of residuals: \(DW\)=2.08, \(p\)=0.33), and normality of error term (Shapiro-Wilk test indicated no evidence against the null hypothesis that the residuals were drawn from a normally distributed population: \(W\)=1, \(p\)=0.762).

Figure 2: Residuals vs Fitted plot demonstrating overall near constant mean and variance of error term across levels of the response

7: Report your test or model results

• Provide a table of results if applicable (for regression tables, try tab_model() from the sjPlot package).
  
• Provide plots if applicable.

Solution

Full regression results including 95% Confidence Intervals are shown in Table 2. The interaction between social comparison and neuroticism in predicting depression and anxiety is visually presented in Figure 3. The F-test for model utility was significant (F(7,647)=35.76, p<.001), and the model explained approximately 27.1% of the variability in DASS-21 Scores.

Table 2: Regression table for DASS-21 model. Outcome variable is raw total score on DASS-21, all predictors are Z-scored

	DASS-21
Predictors	Estimates	CI	p
(Intercept)	44.98	44.53 – 45.42	<0.001
Social Comparison Scale	-1.94	-2.39 – -1.49	<0.001
Neuroticism	1.42	0.97 – 1.86	<0.001
Openness	-0.31	-0.75 – 0.12	0.155
Conscientiousness	0.09	-0.35 – 0.53	0.685
Extraversion	0.53	-0.15 – 1.20	0.124
Agreeableness	0.34	-0.33 – 1.01	0.324
Social Comparison Scale : Neutoricism	-2.77	-3.24 – -2.29	<0.001
Observations	655
R2 / R2 adjusted	0.279 / 0.271

Figure 3: Predicted DASS-21 score across SCS scores, for +/-1 SD Neuroticism

Tell

Communicate your findings

8: Interpret your results in the context of your research question.

• What do your results suggest about your research question?
• Make direct links from hypotheses to models (which bit is testing hypothesis)
• Be specific - which statistic did you use/what did the statistical test say? Comment on effect sizes.
• Make sure to include measurement units where applicable.

Solution

Results showed a significant conditional association between SCS scores (Z-scored) and DASS-21 Scores (\(\beta\) = 44.98,SE = 0.23, p <.01), suggesting that for those at the mean level of neuroticism, scores on the DASS-21 decrease by 44.98 for every 1 standard deviation increase in SCS scores. A significant conditional association was also evident between Neuroticism (Z-scored) and DASS-21 Scores (\(\beta\) = -1.94,SE = 0.23, p <.01), suggesting that for those who score the mean on the SCS, scores on the DASS-21 increase by -1.94 for every 1 standard deviation increase in neuroticism. Crucially, the association between social comparison and symptoms of depression and anxiety was found to be dependent upon the level of neuroticism, with a greater negative association between the two for those with high levels of neuroticism (\(\beta\) = -2.77,SE = 0.24, p <.01). This interaction is visually presented in Figure 3.

The results presented here indicate that the association between social comparison and depression and anxiety may depend upon individuals’ levels of neuroticism, with perceived social rank perhaps leading to more symptoms of depression and anxiety for highly neurotic individuals. However, it is important to note that we can make no claims on the directions of these associations from these data - it may be that social comparison leads to more depression and anxiety in neurotic individuals, but also consistent is the view that - for these individuals - higher levels of depression leads to a greater reduction in perceived social rank.

Tying it all together

All the component parts we have just written in the exercises above can be brought together to make a reasonable draft of a statistical report. There is a lot of variability in how to structure the reporting of statistical analyses, for instance you may be using the same model to test a selection of different hypotheses.

The answers contained within the solution box below is just an example. While we hope it is useful for you when you are writing your report, it should not be taken as an exemplary template for a report which would score 100%.
We have also included the RMarkdown file used to create this, which may be useful to see how things such as formatting and using inline R code can be used.

Solution SPOILERS ALERT

You can find find the .Rmd file for the draft below at https://uoepsy.github.io/files/mlr_writeup_example.Rmd. Why not try downloading and compiling it to see how it works?

PLEASE NOTE: A weird issue with hosting .Rmd files like this means that when you download it, it will remove the top bit of meta-data (i.e., the bit which includes author, title etc) meaning it will not compile without a small amount of editing. You can see the .Rmd file more clearly including these first 6 lines here.

Data was obtained from https://uoepsy.github.io/data/scs_study.csv: a dataset containing information on 656 participants, including Z-scores on the 5 personality traits assessed by the Big-Five Aspects Scale (BFAS) (Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism). Participants were also assessed on the Social Comparison Scale (SCS), which is an 11-item scale measuring self-perception (relative to others) of social rank, attractiveness and belonging, and the Depression Anxiety and Stress Scale (DASS-21) - a 21 item measure with higher scores indicating higher severity of symptoms. For both of these measures, only total scores are available. Items in the SCS are measured on a 5-point scale, giving minimum and maximum possible scores of 11 and 55 respectively. Items in the DASS-21 are measured on a 4-point scale, meaning that scores can range from a possible 21 to 84.

All participant data was complete (no missing values), with scores on the SCS and the DASS-21 all within possible ranges (see Table 3). Bivariate correlations show a moderate negative relationship between DASS-21 and SCS scores; a moderate positive relationship between DASS-21 and Neuroticism, and a weak positive correlation between SCS and Neuroticism. Additionally, a strong positive relationship is evident between Extraversion and Agreeableness (see Figure 4).

Table 3: SCS and DASS-21 descriptive statistics

	n	mean	sd	min	max
scs	656	35.77	3.53	27	54
dass	656	44.72	6.76	23	68

Figure 4: Bivariate scatter plots (below diagonal), histograms (diagonal), and Pearson correlation coefficient (above diagonal), of personality trait measures and scores on the SCS and the DASS-21

¹

Analysis

To investigate whether, when controlling for other personality traits, neuroticism moderates the effect of social comparison on symptoms of depression, anxiety and stress, total scores on the DASS-21 were modelled using multiple linear regression. The Z-scored measures on each of the big-five personality traits were included as predictors, along with scores on the SCS (Z-scored) and its interaction with the measure of Neuroticism. Effects will be considered statistically significant at \(\alpha = 0.01\). One observation was excluded from the final analysis as it was judged to be too influential on the model (Cook’s Distance = 2.66).
The final model was fitted to the remaining 655 observations, and took the form: \[ \text{DASS-21} = \beta_0 + \beta_1 \text{O} + \beta_2 \text{C} + \beta_3 \text{E} + \beta_4 \text{A} + \beta_5 \text{N} + \beta_6 \text{SCS} + \beta_7 \text{SCS} \cdot \text{N} + \epsilon \\ \begin{align} \text{Where} \\ & \text{O = Openness} \\ & \text{C = Conscientiousness} \\ & \text{E = Extraversion} \\ & \text{A = Agreeableness} \\ & \text{N = Neuroticism} \\ \end{align} \]

To address the research question of whether neuroticism moderates the effect of social comparison on depression and anxiety, we will consider the hypothesis test that the interaction coefficient is equal to zero, where:

\(H_0: \beta_7 = 0\). The interaction between SCS and Neuroticism is equal to zero.
\(H_1: \beta_7 \neq 0\). The interaction between SCS and Neuroticism is not equal to zero.

The regression model met assumptions of linearity (see plot of model residuals vs fitted values, Figure 5), homoscedasticity (non-constant variance test indicated no evidence against the null hypothesis that the error variance is constant across level of the response, \(\chi^2(1)\)=1.51, \(p\)=0.219), independence of errors (Durbin-Watson test for autocorrelation of residuals: \(DW\)=2.08, \(p\)=0.33), and normality of error term (Shapiro-Wilk test indicated no evidence against the null hypothesis that the residuals were drawn from a normally distributed population: \(W\)=1, \(p\)=0.762).

Figure 5: Residuals vs Fitted plot demonstrating overall near constant mean and variance of error term across levels of the response

Results

Results showed a significant conditional association between SCS scores (Z-scored) and DASS-21 Scores (\(\beta\) = 44.98,SE = 0.23, p <.01), suggesting that for those at the mean level of neuroticism, scores on the DASS-21 decrease by 44.98 for every 1 standard deviation increase in SCS scores. A significant conditional association was also evident between Neuroticism (Z-scored) and DASS-21 Scores (\(\beta\) = -1.94,SE = 0.23, p <.01), suggesting that for those who score the mean on the SCS, scores on the DASS-21 increase by -1.94 for every 1 standard deviation increase in neuroticism. Crucially, the association between social comparison and symptoms of depression and anxiety was found to be dependent upon the level of neuroticism, with a greater negative association between the two for those with high levels of neuroticism (\(\beta\) = -2.77,SE = 0.24, p <.01). This interaction is visually presented in Figure 6.
The F-test for model utility was significant (F(7,647)=35.76, p<.001), with the model explaining approximately 27.1% of the variability in DASS-21 Scores. Full regression results including 95% Confidence Intervals are shown in Table 4.
The results presented here indicate that the association between social comparison and depression and anxiety may depend upon individuals’ levels of neuroticism, with perceived social rank perhaps leading to more symptoms of depression and anxiety for highly neurotic individuals. However, it is important to note that we can make no claims on the directions of these associations from these data - it may be that social comparison leads to more depression and anxiety in neurotic individuals, but also consistent is the view that - for these individuals - higher levels of depression leads to a greater reduction in perceived social rank.

Figure 6: Predicted DASS-21 score across SCS scores, for +/-1 SD Neuroticism

Table 4: Regression table for DASS-21 model. Outcome variable is raw total score on DASS-21, all predictors are Z-scored

	DASS-21
Predictors	Estimates	CI	p
(Intercept)	44.98	44.53 – 45.42	<0.001
Social Comparison Scale	-1.94	-2.39 – -1.49	<0.001
Neuroticism	1.42	0.97 – 1.86	<0.001
Openness	-0.31	-0.75 – 0.12	0.155
Conscientiousness	0.09	-0.35 – 0.53	0.685
Extraversion	0.53	-0.15 – 1.20	0.124
Agreeableness	0.34	-0.33 – 1.01	0.324
Social Comparison Scale : Neutoricism	-2.77	-3.24 – -2.29	<0.001
Observations	655
R2 / R2 adjusted	0.279 / 0.271

---

1. Created with the pairs.panels() function from the psych package if you’re interested.↩︎