Writing-up

LEARNING OBJECTIVES

Write-up and provide interpretation of a linear model analysis with multiple predictors.

In this lab you will be presented with the output from a statistical analysis, and your job will be to write-up and present the results. We’re going to use an example analysis using one of the datasets we have worked with on a number of exercises in previous labs concerning personality traits, social comparison, depression, and anxiety.

The aim in writing should be that a reader is able to more or less replicate your analyses without referring to your R code. This requires detailing all of the steps you took in conducting the analysis.
The point of using RMarkdown is that you can pull your results directly from the code. If your analysis changes, so does your report!

Make sure that your final report doesn’t show any R functions or code. Remember you are interpreting and reporting your results in text, tables, or plots, targeting a generic reader who may use different software or may not know R at all.

Research question and analysis.

Research Question & Analysis Code

Previous research has identified an association between an individual’s perception of their social rank and symptoms of depression, anxiety and stress. We are interested in the individual differences in this relationship.

Specifically:

Controlling for other personality traits, does neuroticism moderate effects of social comparison on symptoms of depression, anxiety and stress?

Data

Below is an overview of the dataset and measures we will be using:

Data
- Number of participants = 656
- Number of measures = 3
Measures & Scoring
- The Big-Five Aspects Scale (BFAS): Assesses 5 personality traits - Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism. BFAS scores were all standardized.
- Social Comparison Scale (SCS): An 11-item scale measuring self-perception (relative to others) of social rank, attractiveness and belonging. Items on the SCS are measured on a 5-point scale, giving minimum and maximum possible scores of 11 and 55 respectively. Low scores are indicative of feelings of inferiority and general low rank self-perceptions.Only total scores are available.
- Depression Anxiety and Stress Scale (DASS-21): A 21 item self-report measure designed to assess the emotional states of depression, anxiety and stress. Items in the DASS-21 are measured on a 4-point scale, meaning that scores can range from a possible 21 to 84. Higher scores indicate higher severity of symptoms. Only total scores are available.

Download link

https://uoepsy.github.io/data/scs_study.csv

Provided Analysis Code

library(tidyverse) # for all things!
library(psych) # good for descriptive stats
library(kableExtra) # useful for creating nice tables
library(car) # for assumption tests
library(sjPlot) # for plotting models

scs_study <- read_csv("https://uoepsy.github.io/data/scs_study.csv")

# standardise scs score
scs_study <- 
  scs_study %>% 
    mutate(
      zscs = (scs-mean(scs))/sd(scs)
    )
#alternatively, you could do zscs = scale(scs, center = TRUE, scale = TRUE)

# the describe() function is from the psych package, and kable() from kableExtra which is used to make a nice table where the values are rounded to 2 decimal places using digits = 2. 
describe(scs_study) %>% 
    kable(digits = 2) %>%
    kable_styling(full_width = FALSE)

	vars	n	mean	sd	median	trimmed	mad	min	max	range	skew	kurtosis	se
zo	1	656	0.09	1.02	0.08	0.08	1.07	-2.82	3.55	6.37	0.15	-0.08	0.04
zc	2	656	0.02	1.00	0.00	0.03	1.03	-3.22	3.08	6.30	-0.08	-0.13	0.04
ze	3	656	0.00	1.00	-0.04	-0.01	1.00	-3.01	2.80	5.81	0.05	-0.13	0.04
za	4	656	0.00	1.00	-0.02	0.01	1.04	-2.94	2.97	5.91	-0.08	-0.04	0.04
zn	5	656	0.00	1.00	-0.21	-0.10	1.00	-1.45	3.35	4.80	0.80	0.04	0.04
scs	6	656	35.77	3.53	35.00	35.59	2.97	27.00	54.00	27.00	0.60	0.96	0.14
dass	7	656	44.72	6.76	44.00	44.62	5.93	23.00	68.00	45.00	0.18	0.33	0.26
zscs	8	656	0.00	1.00	-0.22	-0.05	0.84	-2.48	5.16	7.64	0.60	0.96	0.04

dass_mdl <- lm(dass ~ 1 + zscs*zn + zo + zc + ze + za, data = scs_study)
par(mfrow=c(2,2))
plot(dass_mdl)

# 35 seems to be a very influential point, lets remove it and re-run the model

dass_mdl2 <- lm(dass ~ 1 + zscs*zn + zo + zc + ze + za, data = scs_study[-35, ])

# check assumptions for updated model
par(mfrow=c(2,2))
plot(dass_mdl2)

par(mfrow=c(1,1))

# N.B. we cannot use crPlots for interactions

# Additional diagnostic plots for independence and homoscedasticity

# checking for independence
plot(resid(dass_mdl2))

# alternative check for equal variances (Homoscedasticity) - 
residualPlots(dass_mdl2)

##            Test stat Pr(>|Test stat|)  
## zscs          1.8141          0.07013 .
## zn           -0.5911          0.55467  
## zo            1.7801          0.07553 .
## zc           -0.2403          0.81018  
## ze           -0.9951          0.32004  
## za            0.0725          0.94219  
## Tukey test   -1.6406          0.10089  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

# multicollinearity
vif(dass_mdl2)

##     zscs       zn       zo       zc       ze       za  zscs:zn 
## 1.015133 1.015736 1.013310 1.008235 2.332486 2.342220 1.012475

# model output
summary(dass_mdl2)

## 
## Call:
## lm(formula = dass ~ 1 + zscs * zn + zo + zc + ze + za, data = scs_study[-35, 
##     ])
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -17.1455  -3.8155  -0.0066   3.6905  18.1483 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 44.97703    0.22635 198.708  < 2e-16 ***
## zscs        -1.93818    0.23042  -8.412 2.58e-16 ***
## zn           1.41639    0.22661   6.250 7.44e-10 ***
## zo          -0.31435    0.22056  -1.425    0.155    
## zc           0.09134    0.22515   0.406    0.685    
## ze           0.52695    0.34233   1.539    0.124    
## za           0.33847    0.34281   0.987    0.324    
## zscs:zn     -2.76609    0.24097 -11.479  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 5.733 on 647 degrees of freedom
## Multiple R-squared:  0.279,  Adjusted R-squared:  0.2712 
## F-statistic: 35.76 on 7 and 647 DF,  p-value: < 2.2e-16

Analysis Strategy

Your analysis strategy will contain a number of different elements detailing plans and changes to your plan. Remember, your analysis strategy should not contain any results. You may wish to include the following sections:

Very brief data and design description:
- Give the reader some background on the context of your write-up. For example, you may wish to describe the data source, data collection strategy, study design, number of observational units.
- Specify the variables of interest in relation to the research question, including their unit of measurement, the allowed range (for Likert scales), how they are scored, and if they are factors make sure to list the order of the levels.
Data management:
- Describe any data cleaning and/or recoding.
- Are there any observations that have been excluded based on pre-defined criteria? How/why, and how many?
- * Describe any transformations performed to aid your interpretation (i.e., log transformation, mean centering, standardisation, etc.)
Model specification:
- Clearly state your hypotheses and specify your chosen significance level.
- What type of statistical analysis do you plan to use to answer the research question? (e.g., t-test, simple linear regression, multiple linear regression, etc.)
- In some cases, you may wish to include some visualisations and descriptive tables to motivate your model specification.
- Specify the model(s) to be fitted to answer your given research question and analysis structure. Clearly specify the response and explanatory variables included in your model(s) and remember to describe the coding of categorical variables (i.e., factors) so the reader is aware of any reference levels.
- Detail the steps that you will undertake to ensure that your model(s) do not violate the appropriate assumptions.
- If applicable, detail any required changes/modifications to the model specification to satisfy assumptions. Consider the following: Was there anything you had to do differently than planned during the analysis? Did the modelling highlight issues in your data? Did you have to do anything (e.g., transform any variables, exclude any observations) in order to meet assumptions?

Note that the * used on occasion in the above indicates that you may/should in some cases repeat these steps if you decide to make any modifications to your data (e.g., removing outliers, etc.)

As noted at the start of the lab, one of the main benefits of using RMarkdown is the ability to include inline R code in your document. Try to incorporate this in your write up so you can automatically pull the specified values from your code. If you need a reminder on how to do this, see Lesson 4 of the Rmd Bootcamp: https://uoepsy.github.io//rmd-bootcamp/.

Task 1

Attempt to draft an analysis strategy section based on the above research question and analysis provided.

Example Write-Up of Analysis Strategy Section

Data was obtained from https://uoepsy.github.io/data/scs_study.csv: a dataset containing information on 656 participants, including Z-scores on the 5 personality traits assessed by the Big-Five Aspects Scale (BFAS) (Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism). Participants were also assessed on the Social Comparison Scale (SCS), which is an 11-item scale measuring self-perception (relative to others) of social rank, attractiveness and belonging, and the Depression Anxiety and Stress Scale (DASS-21) - a 21 item measure with higher scores indicating higher severity of symptoms. For both of these measures, only total scores are available. Items in the SCS are measured on a 5-point scale, giving minimum and maximum possible scores of 11 and 55 respectively. Items in the DASS-21 are measured on a 4-point scale, meaning that scores can range from a possible 21 to 84.

All participant data was complete (no missing values), with scores on the SCS and the DASS-21 all within possible ranges.

To investigate whether, when controlling for other personality traits, neuroticism moderates the effect of social comparison on symptoms of depression, anxiety and stress, total scores on the DASS-21 were modelled using multiple linear regression. The Z-scored measures on each of the big-five personality traits were included as predictors, along with scores on the SCS (Z-scored) and its interaction with the measure of Neuroticism. Effects will be considered statistically significant at \(\alpha = 0.05\).

The following model specification will be used: \[ \text{DASS-21} = \beta_0 + \beta_1 \text{O} + \beta_2 \text{C} + \beta_3 \text{E} + \beta_4 \text{A} + \beta_5 \text{N} + \beta_6 \text{SCS} + \beta_7 (\text{SCS} \cdot \text{N}) + \epsilon \\ \\ \begin{align} \text{where} \qquad & \text{O = Openness, z-scored} \\ & \text{C = Conscientiousness, z-scored} \\ & \text{E = Extraversion, z-scored} \\ & \text{A = Agreeableness, z-scored} \\ & \text{N = Neuroticism, z-scored} \\ & \text{SCS = Social Comparison Scale, z-scored} \\ \end{align} \]

One observation (unit 35) was judged to be too influential on the model (Cook’s Distance = 2.66) and as such was excluded from the final analysis, leaving 655 observations.

To address the research question of whether neuroticism moderates the effect of social comparison on depression and anxiety, we will test whether the interaction between SCS and Neuroticism is significant. Formally, this corresponds to testing whether the interaction coefficient is equal to zero:

\[ H_0: \beta_7 = 0 \\ H_1: \beta_7 \neq 0 \]

For the final model we will visually assess the following assumptions using diagnostic plots: linearity (via plot of residuals vs fitted values), independence (with the previous plot and a plot of residuals vs index), equal variances (via a scale-location plot), and normality (via a qqplot of the residuals). We will also check if there is any evidence of multicollinearity by checking the VIF values, where values > 5 will be considered to indicate moderate multicollinearity, and values > 10 severe.

Results

The results section should follow from your analysis strategy. This is where you would present the evidence and results that will be used to answer the research questions and can support your conclusions. Make sure that you address all aspects of the approach you outlined in the analysis strategy.

In this section, it is useful to include tables and plots to clearly present your findings to your reader. It is important, however, to carefully select what is the key information that should be presented. You don’t want to overload the reader with unnecessary information, and you also want to save space in case there is a page limit. Make use of figures with multiple panels where you can.

As a broad guideline, you want to start with the results of an exploratory data analysis, presenting tables of summary statistics and exploratory plots. You may also want to visualise relationships between variables and report covariances or correlations. Then, you should move on to the results from your model. Remember that in the main part of the report you should only interpret and report for models that do not violate the assumptions. You should also interpret all of the results presented, and remember to make reference to and comment on your assumption and diagnostic checks for key models.

Task 2

Attempt to draft a results section based on your detailed analysis strategy and the analysis provided.

Example Write-Up of Results Section

Descriptive statistics are displayed in Table 1. Bivariate correlations show a moderate negative relationship between DASS-21 and SCS scores; a moderate positive relationship between DASS-21 and Neuroticism, and a weak positive correlation between SCS and Neuroticism. Additionally, a strong positive relationship is evident between Extraversion and Agreeableness (see Figure 1).

Table 1: SCS and DASS-21 descriptive statistics
	n	mean	sd	min	max
scs	656	35.77	3.53	27	54
dass	656	44.72	6.76	23	68

Bivariate scatter plots (below diagonal), histograms (diagonal), and Pearson correlation coefficient (above diagonal), of personality trait measures and scores on the SCS and the DASS-21

Figure 1: Bivariate scatter plots (below diagonal), histograms (diagonal), and Pearson correlation coefficient (above diagonal), of personality trait measures and scores on the SCS and the DASS-21

Figure 2: Diagnostic plots.

The model met assumptions of linearity and independence (see top left panel of Figure 2), homoscedasticity (see bottom left panel of Figure 2), and normality (see top right panel of Figure 2) where QQplot shows very little deviation from the diagonal line.

Full regression results including 95% Confidence Intervals are shown in Table 2. The interaction between social comparison and neuroticism in predicting depression and anxiety is visually presented in Figure 3. The F-test for model utility was significant (F(7,647) = 35.76, p<.001), and the model explained approximately 27.1% of the variability in DASS-21 Scores.

Table 2: Regression table for DASS-21 model. Outcome variable is raw total score on DASS-21, all predictors are Z-scored
	DASS-21
Predictors	Estimates	CI	p
(Intercept)	44.98	44.53 – 45.42	<0.001
Social Comparison Scale	-1.94	-2.39 – -1.49	<0.001
Neuroticism	1.42	0.97 – 1.86	<0.001
Openness	-0.31	-0.75 – 0.12	0.155
Conscientiousness	0.09	-0.35 – 0.53	0.685
Extraversion	0.53	-0.15 – 1.20	0.124
Agreeableness	0.34	-0.33 – 1.01	0.324
Social Comparison Scale : Neutoricism	-2.77	-3.24 – -2.29	<0.001
Observations	655
R² / R² adjusted	0.279 / 0.271

Results showed a significant conditional association between SCS scores (Z-scored) and DASS-21 Scores (\(\beta\) = -1.94, SE = 0.23, p < .001), suggesting that for those at the mean level of neuroticism, scores on the DASS-21 decrease by 1.94 for every 1 standard deviation increase in SCS scores. A significant conditional association was also evident between Neuroticism (Z-scored) and DASS-21 Scores (\(\beta\) = 1.42, SE = 0.23,, p<.001), suggesting that for those who score the mean on the SCS, scores on the DASS-21 increase by 1.42 for every 1 standard deviation increase in neuroticism.

Crucially, the association between social comparison and symptoms of depression and anxiety was found to be dependent upon the level of neuroticism, with a greater negative association between the two for those with high levels of neuroticism (\(\beta\) = -2.77, SE = 0.24, p<.001). This interaction is visually presented in Figure 3.

Figure 3: Predicted DASS-21 score across SCS scores, for +/-1 SD Neuroticism

The results presented here indicate that the association between social comparison and depression and anxiety may depend upon individuals’ levels of neuroticism, with perceived social rank perhaps leading to more symptoms of depression and anxiety for highly neurotic individuals.

Discussion

In the discussion section, you should summarise the key findings from the results section and provide the reader with take-home sentences drawing the analysis together and relating it back to the original question.

The discussion should be relatively brief, and should not include any statistical analysis - instead think of the discussion as a conclusion, providing an answer to the research questions.

Task 3

Attempt to draft a discussion section based on your results and the analysis provided.

Example Write-Up of Discussion Section

Tying it all together

All the component parts we have just written in the exercises above can be brought together to make a reasonable draft of a statistical report. There is a lot of variability in how to structure the reporting of statistical analyses, for instance you may be using the same model to test a selection of different hypotheses.

The answers contained above are just an example of how you can structure and write each section. While we hope it will serve as a useful reference when you are writing your report, but it should not be taken as an exemplary template for a report which would score 100%.

On the Learn page, we have also included the RMarkdown file used to create this report, which may be useful to see how things such as formatting and using inline R code can be used.
Why not try downloading and compiling it to see how it works?