Report LMM analyses and results

In DAPR2, you learned to write an Analysis Plan section followed by a Results section. You might have noticed, though, that published academic articles typically go straight from a Methods section (where they discuss the study design, the study procedure, their participants, and the materials) into a Results section (where they report what they found). In DAPR3, we will train you to write something more like an article-style Results section.

This flashcard doesn’t contain an example write-up, because we want you to learn to write your own report instead of using ours as a template. But below you’ll find guidelines and checklists for you to follow, so that you know you’ve included all the important information.

Checklist for a strong LMM-based Results section

Set the scene (so the reader knows what you’re talking about).

• Remind the reader briefly what the study aimed to test (e.g., what the predictions were).
• Explain what factors the study is manipulating and what the outcome variable is.

Describe and show the sample data (so that the reader knows how generalisable your analysis might be).

• Mention any data cleaning / outlier removal you did before computing the descriptives.
• Describe the outcome and predictor variables in your data (e.g., what scale are they on? how were they measured?)
• Describe the grouping variables in your data (see Describe group-structured data).
  • Tip: Tables and plots are easier to understand than listing loads of numbers in the text (and better for your word count too!).
  • Tip: Plots are easier to understand than tables of numbers.

Summarise and visualise the results that address the RQ (so the reader knows the basic pattern of what you found).

• In a sentence or two, describe the overall pattern of results.
  • Tip: No inferential stats have been done yet, so best not to make any claims yet about whether predictions have been supported or not.
• Include a plot of the data (and make sure to refer to it in the text).
  • Tip: If you transform any continuous predictors for analysis, use the transformed version of the predictor in the plot.

Introduce your statistical analysis (so the reader can interpret the numbers you will report).

• Describe the fixed effects of your model, e.g.: “We fit a linear mixed model predicting Y as a function of X1, X2, and their interaction”.
• If fitting a GLM, describe the model family and the link function, e.g.: “Since we are modelling a binary outcome, the model is binomial with a logit link”.
• If categorical predictors, describe contrast coding, e.g.: “X1 has two levels, A (coded as –1) and B (coded as 1)”.
• If continuous predictors have been transformed, describe the transformation, e.g.: “X2 has been z-scored”.
• Describe the random effect structure of the maximal model.
  • One option using “random” terminology (more traditional): “The maximal model includes by-participant random intercepts, by-participant random slopes over X1, and by-stimulus random intercepts.”
  • Another option using “adjustment” terminology (less traditional but clearer): “The maximal model includes by-participant adjustments to the intercept and to the slope over X1, as well as by-stimulus intercept adjustments.”
• If you had to simplify the random effect structure, describe the order in which you removed parameters.
• Describe the random effect structure of the actual model that you will report.
• State that you’re running your analysis using R, and cite the developers (run citation() in the console to get the citation for your version of R).
• State the software packages and versions used to fit models, along with the estimation method (ML/REML), how p-values were approximated (probably Satterthwaite), what optimiser you used…
• State which model coefficient(s) are related to which prediction(s), and what direction/sign (positive or negative) you would expect the coefficients to have if the predictions were borne out.

Report the results of your analysis (so the reader knows exactly what you found).

• State the key coefficient estimates along with their 95% CI and p-value, e.g.: “The slope associating X1 and Y was significantly different from zero (\(b\) = 0.26, 95% CI [0.23, 0.29], \(p\) < 0.001)” or “The slope associating X1 and Y was not significantly different from zero (\(b\) = 0.26, 95% CI [–0.04, 0.56], \(p\) = 0.18)”.
  • Tip: You don’t need to report every single parameter estimate. Focus on the one(s) you identified as being relevant to your prediction(s).
  • Tip: Briefly link the direction and significance of the effect back to your earlier predictions—is the prediction supported by the parameter estimate?
• Include a table of all coefficients estimated by the linear model (and make sure to refer to it in the text).
• Report the magnitude of the SD of the group-level adjustments and compare it to the magnitude of the fixed effects. How does the intercept vary between groups? How does the effect of interest vary between groups?
• Report the correlations between intercept and slope adjustments.
• Plot the fitted model estimates, including group-level estimates (see Plot model-fitted values).
• Report any other information you consider important for your story!

General guidelines

• Your golden guiding principle should be: does the reader have all the information they would need to replicate your analysis in whatever statistical software they might be using?

• The goal of a Results section is to tell the reader what you found. The goal of the Discussion section is to tell the reader what it all means. The actual interpretation of the results belongs in the Discussion section, not the Results section.

• Follow the APA guidelines on how to report numbers and statistics.

• If you make a plot that contains error bars/shaded error ribbons, always tell the reader what they represent (standard deviation? standard error? 95% CI?).

• To make tables look extra professional, use horizontal lines only.

Linked flash cards

Outgoing links

• TODO

Backlinks

• TODO