Two Big Things

I’m providing the code so you can play along at home but there really just two big things to remember.

1. When we “annualize” turnover we convert the results from our given time scale to the annual scale. If you have a monthly turnover rate you multiply that value by 12 to see what that rate would look like over the course of a year. For instance, if you lost 4% in a month than you would expect to lose 48% over the course of a year if you continued at that rate (see this post for more on annualized turnover rates).
2. The importance of the signal-to-noise ratio (SNR). As we learned in the first post, SNR is just the ratio of the mean to the standard deviation. Annualizing our turnover results from quarterly or monthly turnover data dramatically increases the standard deviation and therefore the random swings in our results. Cranking up the noise means lowering the SNR, and, ultimately, big movements in the reported turnover results without any real change in the underlying system.

The aim here is to show you how this happens to all of us everyday when we are scrambling to get our latest HR data into the hands of leaders.

Now on to the results:

# Monte Carlo Annualized Turnover
library(plyr)
library(dplyr)
library(knitr)

# Creating the matrix
set.seed(42)
obs <- 1000000
sets_ann <- matrix(data = NA, nrow = obs, ncol = 3)

for (i in c(1:3)){
  time <- c(1,4,12)[i]
  p <- 1000
  pr <- .1/time #calculating number of expected quits based on time unit

  # Multiplying the result by the time factos is what I need to "annualize" it
  sets_ann[,i]<- rbinom(obs, p, pr)*time # Get annualized rate

}

sets_df <- data.frame(Annually = sets_ann[,1], Quarterly = sets_ann[,2], Monthly = sets_ann[,3])

# Get SNRs of the raw frequent data ---------------------------------------

SNR <- lapply(sets_df[,1:3], FUN = function(x){
  temp_df <- data.frame(round(mean(x), 2), round(sd(x), 2), round(mean(x)/sd(x), 2))
})

SNR <- ldply(SNR)
names(SNR) <- c("Frequency", "Ann_Mean", "SD", "SNR")
#row.names(SNR) <- c("Annual", "Quartely", "Monthly")

#kable(SNR[,2:4], align = 'l')
kable(SNR)

Monte Carlo Results: By the Numbers

What do our results show us? First, note that our average annualized results all give 100 as we would expect. That is, on average, we are losing 100 people per year regardless of whether we calculate that based on annual, quaterly, or monthly data. This is just a sanity check that we are simulating what we think we are simulating.

The key thing we see is a dramatic increase in the standard deviation of our results. Just moving from annual to quarterly reporting doubles our SD and cuts our SNR in half. Moving from quarterly to monthly reporting of annualized turnover is similarly damaging.

What this means in practice is that we’ll end up many more “good” periods of turnover but also many more “bad” periods of turnover as we increase reporting frequency. Whether we’re patting ourselves on the back or gnashing our teeth, we’ll often overreact based on solely on more noisy data and not any real underlying change.

Monte Carlo Results: By the Pictures

A picture is worth a thousand tables so let’s use some histograms. The standard definition of an outlier is a result that is roughly +/- 2 SDs from the mean. We know that the SD for our true annual Monte Carlo results is 9.5. We’ll draw that histogram with 2 red bars to highlight the boundaries for these outlier values. About 5% of our true annual observations fall into either of these outlier areas.

set.seed(42)
temp_samp <- sets_df[sample(nrow(sets_df), size = 8000), ]

# Getting the Outliers for the annual distributions
lout_2 <- mean(sets_df$Annually) - 1.96*sd(sets_df$Annually)
rout_2 <- mean(sets_df$Annually) + 1.96*sd(sets_df$Annually)

hist(temp_samp[,1], breaks = 20, xlim = c(0,250), main = "Annual")
abline(v = lout_2, col = 'red', lwd = 3)
abline(v = rout_2, col = 'red', lwd = 3)
mtext(text = '-2 SDs', side = 1, line = .5, at = lout_2, cex = .7)
mtext(text = '+2 SDs', side = 1, line = .5, at = rout_2, cex = .7)

Now we’ll use those same outlier values but apply them to the annualized QUARTERLY turnover. When we do this, a funny thing happens: we end up with 36% of our annualized quarterly turnover values falling into the outlier category, not 5%.

These extreme results are simply due to noise but I assure you that won’t be the interpretation when we present the results to senior leadership.

hist(temp_samp[,2], breaks = 20, xlim = c(0,250), main = "Quarterly Annualized")
abline(v = lout_2, col = 'red', lwd = 3)
abline(v = rout_2, col = 'red', lwd = 3)
mtext(text = '-2 SDs', side = 1, line = .5, at = lout_2, cex = .7)
mtext(text = '+2 SDs', side = 1, line = .5, at = rout_2, cex = .7)

It gets worse with annualized MONTHLY turnover numbers: a full 60% of our annualized observations fall into the outlier range.

hist(temp_samp$Monthly, breaks = 20, xlim = c(0,250), main = "Monthly Annualized")
abline(v = lout_2, col = 'red', lwd = 3)
abline(v = rout_2, col = 'red', lwd = 3)
mtext(text = '-2 SDs', side = 1, line = .5, at = lout_2, cex = .7)
mtext(text = '+2 SDs', side = 1, line = .5, at = rout_2, cex = .7)