W7: Z-scores & the Normal Distribution

Pick someone new to be the driver this week. If everyone in the group has already had a week as the driver, start going round again!

The driver will type the code. The rest of you are navigators, you need to tell the driver where to go (i.e., what to type!).

Driver instructions:

  1. Log in to the bigger screen at your table
  2. Open RStudio Server. See the first drop-down in the FAQs page for instructions
  3. Open a new .R script, and load the tidyverse package.
  4. Listen to your navigators

Navigator instructions:

  1. Tell your driver where to go (i.e., what to type)!
Question 1

In this week’s workshop, you will practice transforming raw scores to z-scores, reasoning about the probability of those z-scores, and back-transforming z-scores onto their original scale. The goal of this workshop is to help you get comfortable with the concept of transforming variables into a standardised score.

Once again, read in the survey data from https://uoepsy.github.io/data/dapr1_2627_survey.csv. Store it under the variable name sdata.

🗂️ See Read in and inspect data flash card.

This week, we’ll work with the data about people’s outlook (how pessimistic vs. optimistic they are) and their chronotype (how much of an evening person vs. a morning person they are). Here are the questions that generated that data:

You’ll notice that each question uses a quite different scale: outlook ranges from –100 to 100, while ampm (the chronotype variable) ranges from 0 to 10. So someone with an outlook score of 10 is in a very different part of the scale, compared to someone with an ampm score of 10!

Create two new variables in your dataframe. One should be the z-scored version of outlook (call it outlookZ), and the other should be the z-scored version of ampm (call it ampmZ).

🗂️ See Mutate columns and Calculate z-scores flash card.

library(tidyverse)

sdata <- read_csv("https://uoepsy.github.io/data/dapr1_2627_survey.csv")

sdata <- sdata |>
  mutate(
    outlookZ = ( outlook - mean(outlook) ) / sd(outlook),
    ampmZ = ( ampm - mean(ampm) ) / sd(ampm)
  )

Question 2

In this question, you’ll find the pseudonyms, scores, and z-scores of the three most optimistic people and the three “morning people”-est people.

First, sort your dataframe by your outlookZ column, in descending order.

🗂️ See Sort a dataframe with arrange() flash card.

Next, extract the top three rows of your sorted dataframe.

🗂️ See Subset a dataframe > Extract row(s) flash card.

Finally, extract only the columns that we care about for this question: pseudonym, outlook, and outlookZ.

🗂️ See Subset a dataframe > Extract columns(s) > select() flash card.

Repeat this process for the ampm variable to find the three people who identify most strongly as morning people.

Is any of these people you?

Here’s how we’d do all of these steps, piped together into a single statement for outlook and another one for ampm.

sdata |> 
  arrange(desc(outlook)) |>
  slice(1:3) |>
  select(pseudonym, outlook, outlookZ)
# A tibble: 3 × 3
  pseudonym outlook outlookZ
  <chr>       <dbl>    <dbl>
1 <NA>          100     1.70
2 mj            100     1.70
3 alex          100     1.70
sdata |> 
  arrange(desc(ampm)) |>
  slice(1:3) |>
  select(pseudonym, ampm, ampmZ)
# A tibble: 3 × 3
  pseudonym  ampm ampmZ
  <chr>     <dbl> <dbl>
1 <NA>         10  2.32
2 bartlett     10  2.32
3 <NA>         10  2.32

Question 3

From now on, we’ll just focus on outlook and its z-scores. *We’ll come back to ampm in the last workshop of this semester.)

Plot your standardised outlook scores (that is, outlookZ) as a density curve.

🗂️ See Make a density plot flash card.

Once you have made your plot, add this bit of code to the end of it. This additional code will overlay a red standard normal distribution (with mean 0, standard deviation 1) over your density curve. (The ...s in the code chunk represents the code you used to make the density plot.)

... 
  ... +
  stat_function(fun = dnorm, args = list(mean = 0, sd = 1), colour = "red")

You don’t have to know or remember any of this! But if you’re curious:

  • stat_function() adds a statistical function to the plot. This is different from adding a “geom” because geoms are based on data—statistical functions are calculated based on the information we give next.
  • fun = dnorm specifies that the function (fun) to add is the normal (norm) distribution (d).
  • args = list(mean = 0, sd = 1) specifies that the function should use a mean of 0 and a standard deviation of 1, which are the parameters of the standard normal distribution.
  • colour = "red" tells R that we want the distribution to be coloured red.

ggplot(sdata , aes(x = outlookZ)) +
  geom_density() +
  stat_function(fun = dnorm, args = list(mean = 0, sd = 1), colour = "red")

Question 4

This is a challenging question, get ready!

To start: Did anyone in your group give a pseudonym in the survey? If yes, use your groupmate’s pseudonym for this question; if no, use “noodle_horse” (that’s what Josiah calls his greyhound Dougal).

Next, extract the row containing that pseudonym from the dataset using the tidyverse function filter().

🗂️ See Subset a dataframe > filter() flash card.

Now: using the function pnorm(), compute the probability of seeing somebody less optimistic than this person. In other words, you want to find the probability that somebody has a lower outlookZ score than your chosen person (or greyhound).

Next: still using pnorm(), compute the probability of seeing somebody more optimistic than your chosen person (or greyhound).

🗂️ See Explore a normally-distributed variable flash card.

Finally: Check that these two probabilities sum up to 1.

Dougal the noodle horse is a little bit of a pessimist:

sdata |> 
  filter(pseudonym == "noodle_horse") |>
  select(pseudonym, outlook, outlookZ)
# A tibble: 1 × 3
  pseudonym    outlook outlookZ
  <chr>          <dbl>    <dbl>
1 noodle_horse      -7   -0.629

For convenience, let’s extract his outlookZ value and save it under its own name.

dougalZ <- sdata |>
  filter(pseudonym == "noodle_horse") |>
  pull(outlookZ)

To compute the probability of observing a z-score that’s lower than Dougal’s z-score of -0.63, we give this z-score as an argument to pnorm() (see the flash card for helpful illustrations!).

pnorm(dougalZ)
[1] 0.2648

There’s a 26.48028% probability of seeing someone less optimistic than Dougal.

And on the flip side, to compute the probability of observing a z-score that’s higher than Dougal’s z-score, we subtract the output of pnorm() from 1:

1 - pnorm(dougalZ)
[1] 0.7352

There’s a 73.51972% probability of seeing someone more optimistic than Dougal.

These values do indeed sum to 1, and this should make a little bit of sense: for anyone else that you sample, they will either be more optimistic than Dougal or less optimistic than Dougal. So the probability of seeing someone either more or less optimistic than Dougal is 100%.

Question 5

Finally, z-scoring is not just a one-way process. Sometimes you might start with variables in z-scored form, and you might want to transform them back onto their original scale (in other words, you might want to “back-transform” them).

Here’s the process that will “undo” a z-score by back-transforming it onto the original scale:

\[ \text{value on original scale} = \text{original mean} + (\text{z-score} \times \text{original SD}) \] Here’s the more mathematical version of this formula from the lecture slides:

\[ X = M + (Z \times SD) \]

where \(X\) is the raw score / the score on the original scale, \(M\) is the population mean (or our best guess about it based on our sample), \(Z\) is the z-score, and \(SD\) is the population standard deviation (or our best guess about it based on our sample).


First, to get the parts of this formula that we need, run the following code to find the original mean and SD of outlook and store them under their own variable names:

outlook_mean <- mean(sdata$outlook)
outlook_sd   <- sd(sdata$outlook)

Using the formula above, calculate the value on the original outlook scale that corresponds to a z-score of 1. (In other words, what value of outlook is one standard deviation above the mean?)

Finally, calculate the value on the outlook scale that corresponds to a z-score of –2. (In other words, what value of outlook is two standard deviations below the mean?)

🗂️ See Calculate z-scores > Back-transform flash card.

Back-transforming from a z-score of 2, the outlook score that’s one SD above the mean is:

outlook_mean + (1 * outlook_sd)
[1] 67.689


Back-transforming from a z-score of –1, the outlook score that’s two SDs below the mean is:

outlook_mean + (-2 * outlook_sd)
[1] -69.893

Finishing up

Once you’ve finished, don’t forget to:

  1. Save your R script
  2. Post the R script to the group discussion space on Learn, to ensure that everyone has access to it! (For step-by-step instructions on how to do this, click here)