Report LMM analyses and results

In DAPR2, you learned to write an Analysis Plan section followed by a Results section. You might have noticed, though, that published academic articles typically go straight from a Methods section (where they discuss the study design, the study procedure, their participants, and the materials) into a Results section (where they report what they found). In DAPR3, we will train you to write something more like an article-style Results section.

This flashcard doesn’t contain an example write-up, because we want you to learn to write your own report instead of using ours as a template. But below you’ll find guidelines and checklists for you to follow, so that you know you’ve included all the important information.

Checklist for a strong LMM-based Results section

Set the scene (so the reader knows what you’re talking about).

• Remind the reader briefly what the study aimed to test (e.g., what the predictions were).
• Introduce the variables the study is using to test this prediction, but avoid using R variable names; describe the variables in human-readable terms instead (e.g., what scale are they on? how were they measured?)

Describe and show the sample data (so that the reader knows how generalisable your analysis might be).

• If applicable, mention any data cleaning / outlier removal you did before computing the descriptives.
• Describe the data set: how many observations?
• Describe the grouping variables in your data (see Describe group-structured data).
  • Tip: If you find yourself listing loads of numbers in the text, consider using tables or plots instead (and they’re better for your word count too!).

Visualise and summarise the results that address the RQ (so the reader knows the basic pattern of what you found).

• Include a plot of the data (and make sure to refer to it in the text).
  • Tip: If you transform any continuous predictors for analysis, use the transformed version of the predictor in the plot.
  • Tip: Academic writing styles vary, but Elizabeth likes it when figure captions give the reader a brief take-home message about what they should see in the plot.
• In a sentence or two, describe the overall pattern of results.
  • Tip: No inferential stats have been done yet, so we can’t make any claims just yet about whether the predictions were supported. But it is OK to say whether the results appear broadly in line with the prediction.

Introduce your statistical analysis (so the reader can interpret the numbers you will report).

• State that you’re running your analysis using R, and cite the developers (run citation() in the console to get the citation for your version of R).
• Describe the fixed effects of your model, e.g.: “We fit a linear mixed model predicting Y as a function of X1, X2, and their interaction”.
• If fitting a GLM, describe the model family and the link function, e.g.: “Since we are modelling a binary outcome, the model is binomial with a logit link”.
• If categorical predictors, describe contrast coding, e.g.: “X1 has two levels, A (coded as –1) and B (coded as 1)”.
• If continuous predictors have been transformed, describe the transformation, e.g.: “X2 has been z-scored”.
• Describe the random effect structure of the maximal model.
  • One option using “random” terminology (more traditional): “The maximal model includes by-participant random intercepts, by-participant random slopes over X1, and by-stimulus random intercepts.”
  • Another option using “adjustment” terminology (less traditional but clearer): “The maximal model includes by-participant adjustments to the intercept and to the slope over X1, as well as by-stimulus intercept adjustments.”
• If you had to simplify the random effect structure, describe the order in which you removed parameters.
• Describe the random effect structure of your final model that converged without singularities (that is, describe the model that you’ll actually report below).
• State the model’s estimation method (REML or ML), how p-values were computed (if lmerTest, probably Satterthwaite’s method), what optimiser you used, …

Report the results of your analysis (so the reader knows exactly what you found).

• State which model coefficient(s) are related to which prediction(s), and what direction/sign (positive or negative) you would expect the coefficients to have if the predictions were borne out.
• State the key coefficient estimates along with their 95% CI and p-value, e.g.: “The slope associating X1 and Y was significantly different from zero (\(b\) = 0.26, 95% CI [0.23, 0.29], \(p\) < 0.001)” or “The slope associating X1 and Y was not significantly different from zero (\(b\) = 0.26, 95% CI [–0.04, 0.56], \(p\) = 0.18)”.
  • Tip: You don’t need to report every single parameter estimate. Focus on the one(s) you identified as being relevant to your prediction(s).
  • Tip: Briefly link the direction and significance of the effect back to your earlier predictions—is the prediction supported by the parameter estimate?
• Include a table of all coefficients estimated by the linear model (and make sure to refer to it in the text). The function tab_model() from the package sjPlot is useful here, but note that tab_model() causes Rmd files to fail to knit to PDF, so you might want to manually recreate the table it produces.
• Plot the model-fitted values (and refer to the plot in the text).
• Report the magnitude of the SD of the group-level adjustments and compare it to the magnitude of the fixed effects. How does the intercept vary between groups? How does the effect of interest vary between groups?
• Report the correlations between intercept and slope adjustments.
• Report any other information you consider important for your story!

General guidelines

• Your golden guiding principle should be: does the reader have all the information they would need to replicate your analysis in whatever statistical software they might be using?

• The goal of a Results section is to tell the reader what you found. The goal of the Discussion section is to tell the reader what it all means. The actual interpretation of the results belongs in the Discussion section, not the Results section.

• Follow the APA guidelines on how to report numbers and statistics.

• If you make a plot that contains error bars/shaded error ribbons, always tell the reader what they represent (standard deviation? standard error? 95% CI?).

• To make tables look extra professional, use horizontal lines only.

Linked flash cards

Outgoing links

• TODO

Backlinks

• TODO