Error Bars

I. Rule one

It is a crime to plot measures of central tendency without an indication of their variability. Enough said!

II. What do we use as errorbars?

There are pretty much two options: standard errors, or confidence intervals. These quantities are related. The confidence interval is the standard error multiplied by the critical value of a test statistic, which is either t or Z, depending on whether we know the population parameters or estimate them from a sample. The choice really depends upon your rhetorical intent: different things can be concluded from the errorbars, depending on what you choose to plot.

Standard errors
From an overlap, you can conclude no significant difference
Approximately 68% confidence interval for population mean
Difference between means is hard to evaluate
Confidence intervals
Can't draw conclusions from overlap
Exact confidence interval for population mean
Difference between means from multiplying by root 2
Most papers I've read recently plot standard errors. I suspect an ulterior motive...

III. Errorbars for between-subject means

We have two ways of estimating the standard error: a local and a global estimate. Again, it's up to you which one you use. If you're going to be using within-subjects errorbars subsequently, then it's best to use the global estimate for consistency.

Local estimate of the standard error



Global estimate of the standard error



Remember to multiply by the critical value of your test-statistic if you want confidence intervals!

IV. Errorbars for within-subject means

The trick is to think about what is the best estimate of the error variance. When you do a within-subjects ANOVA, the analogue of the MSE is the mean square for the interaction of subjects and the effect you're testing. Basically, if you want to show differences between means on the basis of some factor, replace the MSE in the equation for between-subject means with whatever appears in the denominator of your within-subjects F-ratio.

V. Errorbars for categorical data

Binomial data
How do we work out the confidence interval on an estimate of the probability of an event? Let's say our estimate is p. What's the confidence interval on p?

In general, we have

where q = (1-p).

Multinomial data
It seems like things should get more complicated when we have more than two options. In fact, they don't. We work out the standard error in exactly the same way. There's a simple reason why this is true. Say we're interested in putting errorbars on p1, the estimate of the probability that events fall into category 1. Then we can divide all of our categories up into two kinds: category 1, and everything else. The probability of being in the first category is, of course, p1. But this is exactly the same as the binomial case, which we already know how to deal with! Since we haven't done anything to p1, our estimate of the standard error on p1 remains the same.

Written by Tom Griffiths