The Intuition

To understand the standard deviation, let’s start with the average (or the mean).

Suppose we have two groups of people. In the first group, we have 3 people who have taken 11, 13, and 15 unscheduled days off of work, giving us an average of 13 days. In our second group, we have 3 people who have taken 0, 13, and 26 unscheduled days off of work, again giving us an average of 13 days.

As I have noted in previous posts here and here, the mean is a useful measure because it summarizes all of the values we have into a single value.

We can then use that single values to compare groups.

Here then, we see that both groups have a mean of 13 days. But your intuition is telling you that these groups are different and your intution is right: those in the first group all have similar values while those in this second group are all over the place.

The problem is that averages tell us nothing about the spread or closeness of the values within those groups. For that we need to the standard deviation.

In essence, the standard deviation tells us how close our scores are to the group average. If the standard deviation is low, the scores are generally close to the mean and therefore close to one another. If the standard deviation is high, the individual scores are quite different from the mean and from one another.

The scores from Group 1 (11, 13, and 15) are less spread out than the scores from Group 2 (0, 13, 26). Accordingly, the standard deviation for Group 1 will be less than that for Group 2.

Let’s now move beyond glances and intuition.

The Calculation Steps and What They Mean

Here we show you the individual steps for calculating the standard deviation. Normally, you would not be doing this by hand, but stepping through the logic by hand will help you understand what the standard deviation really means. We’ll show the sd() function in a bit.

Step 1. Calculate the mean of the scores for your group. Think of the mean as the reference point.

g1 <- data.frame(days = c(11, 13, 15)) # assigning the values to group 1
mean(g1$days)

## [1] 13

Step 2. Subtract each of the individual scores from the mean. This is tells us how far each individual value is from that mean reference point.

g1$diff <- g1$days - mean(g1$days) # Step 2: get the difference
g1$diff

## [1] -2  0  2

Step 3. Square those difference scores and then add them all up. We are trying to get a total measure of all of the differences. If just added them up without squaring them first though, the positive and negative values would just cancel out.

g1$diffsq <- g1$diff^2 # squaring each of those values
g1$diffsq
tot_diff <- sum(g1$diffsq) 
tot_diff

## [1] 4 0 4
## [1] 8

Step 4. Divide that by one less than the number of scores you have. Here, we have three scores so we divide our total by 2 (from 3-1 = 2). This is like getting an average of those difference scores. If we had 100 values, we would divide by 99 (from 100-1 = 99).

variance <- tot_diff/2 # Step 5: 3 total values minus one so 3-1 =2. 
variance

## [1] 4

Step 5. Take the square root of that value we obtained in Step 4. In this step we are essentially undoing that squaring business in Step 3 so we have a number that we can interpret.

stdev <- (variance)^.5 # Step 6: get the square root of the variance 
stdev

## [1] 2

Our final calculated standard deviation for Group 1 is 2.

Comparison with Group 2 Standard Deviation

Now let’s see what the standard deviation tells us when apply these same steps to Group 2:

g2 <- data.frame(days = c(0, 13, 26 )) #unscheduled days off
mean(g2$days) # Step 1: calculate the mean 
g2$diff <- g2$days - mean(g2$days) # Step 2: get the difference
g2$diff
g2$diffsq <- g2$diff^2 # Step 3: square those difference.
g2$diffsq
tot_diff <- sum(g2$diffsq) # Step 4: square the summed differences
variance <- tot_diff/2 # Step 5: 3 total scores less one so 3-1 =2. 
stdev <- (variance)^.5 # Step 6: get the square root of the variance 
stdev

## [1] 13
## [1] -13   0  13
## [1] 169   0 169
## [1] 13

Here we get a standard deviation of 13, not 2. This makes sense because the spread of the scores in Group 2 was much greater than the spread from Group 1. The standard deviation captures this difference.

Now that you understand what the standard deviation is, let’s explore it R.