No mean feat

Physicist Stephen Hawking turned 70 last weekend, and has been living with ALS — amyotrophic lateral sclerosis — for nearly 50 years. Usually, the disease is diagnosed in patients over 50, and they die within a few years. I was reading an article in Scientific American about Dr Hawking’s longevity. The article contains an edited interview with Dr Leo McCluskey, an ALS expert at the University of Pennsylvania.

One answer, in particular, struck me:

Sci Am: What has Stephen Hawking’s case shown about the disease?

Dr McCluskey: One thing that is highlighted by this man’s course is that this is an incredibly variable disorder in many ways. On average people live two to three years after diagnosis. But that means that half the people live longer, and there are people who live for a long, long time.

The mathematician in me rose up at that: no, on average does not mean that half the samples are on each side of the average. Average refers to the arithmetic mean — take a bunch of numbers, add them, and divide by the count (how many numbers you added) — and it’s easy to show, by example, how that’s wrong.

Suppose we had five patients with ALS. Suppose four of those patients lived for one year following diagnosis, and one lived for eleven years. 1 + 1 + 1 + 1 + 11 = 15, and 15 / 5 = 3. So on average, people in this sample lived for three years... and only one of the five (20%) survived more than even one year. Given Dr Hawking’s experience of on the order of 50 years, he could offset about 25 patients who succumbed after one year, and still give us a three-year average.

The problem with the arithmetic mean is that it’s easily skewed by outliers. In the extreme example here, if 96% of the samples are 1 and 4% are 50, we get an average of 3 — three times the normal value. That means that with such a situation, the average is useless in giving us any reasonable prediction of what to expect. More generally, if the numbers are widely variable, the average doesn’t tell us anything useful. If we have nine patients who made it through 1, 2, 3, 4, 5, 6, 7, 8, and 9 years, respectively, what do we tell the tenth patient who shows up? 5 years, on average, sure, but, really, we might as well tell him to take a wild guess.

Averages are useful when the values tend to cluster around the arithmetic mean, particularly when the number of samples is large. They’re also helpful in analyzing trends, when we look at the change in the average over time... but, again, we have to be careful that a new outlier hasn’t skewed the average. Sometimes we adjust averages to try to compensate for the outliers — for example, we might eliminate the top and bottom 5% of the samples before taking the average.

Another common error is to confuse the mean with the median. The latter is often used in financial reporting: median income, median purchase price for houses, and so on. The median is a completely different animal from the mean. It’s, quite simply, the middle value. List all the sample values in increasing order, and pick the one in the middle (or one of the two in the middle, if the number of values is even).

In the first example above,^[1] if we write the values as 1, 1, 1, 1, 11, the median is the value in bold: 1. In the second example, we have 1, 2, 3, 4, 5, 6, 7, 8, 9, for a median of 5. You can see that in the first case, the median is not related to the mean, while in the second case it’s the same as the mean. It’s also the case that the mean (or average) is an artificial value that might not appear in the samples, whereas the median is, by definition, one of the sample values.

Also by definition, at least half the sample values are greater than or equal to the median (and at least half are less than or equal to it). In other words, Dr McCluskey’s statement would have been true (at least close enough) had he been talking about the median survival period, rather than the average. Medians are also less susceptible to skewing by outliers, as you can see from the first example.

But as the second example shows, when the numbers are all over the place, neither is of much use in predicting anything.

---

^[1] My examples use small numbers of values for convenience. In reality, both mean and median require a fairly large sample size to be useful at all.