Which election was the closest?
Last night, there was a very close byelection in Runcorn and Helsby. The result was
| Reform | Labour |
|---|---|
| 12,645 | 12,639 |
Very close! (In this and all results, I’ll only look at the top two candidates, and ignore any other votes.)
But was this closer than the most famous close election of recent times: the 2000 US presidential election in Florida? The result there was:
| Bush | Gore |
|---|---|
| 2,912,790 | 2,912,253 |
So which was closer? Well, that depends. How do we measure the closeness of elections?
The simplest way would be just to look at the absolute difference or the “winning margin” d: that is, the number d of extra votes the winning candidate got over the losing candidate. Smaller differences are more impressively tight elections: a small absolute difference means a very close election, while a big absolute difference is not a close election. So how does that work out here?
| Reform | Labour | abs. diff. |
|---|---|---|
| 12,645 | 12,639 | 6 |
| Bush | Gore | abs. diff. |
|---|---|---|
| 2,912,790 | 2,912,253 | 537 |
Well, that solves it! The Runcorn byelection was decided by 6 votes, while Florida 2000 was decided by 537 votes. So Runcorn was much, much closer!
But wait. These two elections were of very different sizes. There were about 25,000 voters in Runcorn, between the top two candidates, but almost 6 million voters in Florida! Surely that needs to be taken into account? Otherwise, we could say that Manchester United’s 3–0 thrashing of Atlético Madrid last night (absolute difference: 3) was even closer than the byelection.
So perhaps it makes more sense to look at the relative difference – that is, by what percentage one candidate beat the other one, or the absolute difference d divided by the total number of votes n. (Again, n will still just be the total number of votes for the top two candidates.)
| Reform | Labour | abs. diff. | rel. diff. |
|---|---|---|---|
| 12,645 | 12,639 | 6 | 0.024% |
| Bush | Gore | abs. diff. | rel. diff. |
|---|---|---|---|
| 2,912,790 | 2,912,253 | 537 | 0.009% |
So, both very close elections. But smaller values of relative difference are still better, so this time, it’s Florida 2000 that wins: the relative difference is reduced by a factor of two-and-a-bit compared to Runcorn. So Florida was closer!
Looking again at relative difference
I think this is the result that most people would accept: the closeness of elections is decided by relative (or percentage) difference between the top two candidates, so Florida 2000 was closer.
But I want to think a bit more carefully about the relative difference: What does it mean, and what justification can we give for its use?
I propose that one way to think about the relative difference is the following.
Suppose we have 99 voters. Then there are 100 possible numerical results, from a 99–0 wipe-out for Red over Blue, to a 98–1 trouncing for Red over Blue, then 97–2, all the way through to a 0–99 wipe-out for Blue over Red. Suppose further that all 100 of those results are equally likely, coming up with 1% probability each. We could then come up with a kind of p-value: What is the probability (under this model) that the election result was as close as this or closer?
Suppose in our example that the Red party wins 51–48. Then, out of the 100 possible outcomes, there are four outcomes that would be as close as this or closer: 51–48 itself, plus 50–49, 49–50, and 48–51. (In the latter two, Blue beat Red.) So the “p-value” (or “somewhat p-value-like quantity”) is 4/100, or 0.04.
So now consider an election with n voters and a difference of d. What’s the p-value here? The answer is that there are d + 1 outcomes as close as this or closer, and n + 1 possible outcomes. So the answer is (d + 1) / (n + 1).
[Optional proof: To see this, we need to count up all the results between “this result” and “the opposite result”, where the other candidate wins by the same amount. To count up these outcomes, we can imagine distributing the winner’s votes to the loser: we could distribute 0 (current result) or 1 (two votes closer in the margin – unless there was only one vote in it to start with), or 2 votes, or 3, up to d votes; once we have distributed d votes, we get the “reverse” result, with the same “closeness” as the original result; any further distribution gives a less-close result with a bigger victory for the originally-losing candidate.]
This p-value we have obtained, (d + 1) / (n + 1), is very nearly the absolute difference d/n; there’s just an extra “plus 1” on the top and bottom. So, my first proposal is not to look at the relative difference d/n, but rather the adjusted relative difference (d + 1) / (n + 1).
For all but the tiniest electorates, replacing n by n + 1 makes almost no difference. For extremely close elections, though, replacing d by d + 1 can affect things: this is a slight extra penalty for the smaller election. (Like Runcorn!)
| Reform | Labour | rel. diff. | adj. rel. diff. |
|---|---|---|---|
| 12,645 | 12,639 | 0.024% | 0.028% |
| Bush | Gore | rel. diff. | adj. rel. diff. |
|---|---|---|---|
| 2,912,790 | 2,912,253 | 0.00922% | 0.00924% |
Here, our adjustment slightly increased the score of the smaller Runcorn byelection, but barely changed the much bigger Florida election. Now, the Florida 2000 score is reduced by about a factor of 3 compared to Runcorn.
To put it another way, under this “each result equally likely” model:
- a result like the Runcorn byelection would happen once every 3,600 elections;
- a result like Florida 2000 would happen once every 11,000 elections.
So the Runcorn byelection result is (under this model) about three times as likely as the Florida 2000 result.
Before we move on, one last argument for the adjusted relative difference, with the “plus 1”s, over the standard relative difference. Consider two elections: in the first, there are four voters, and it’s a 2–2; in the second, there are four million voters, and, astonishingly, it’s an exact 2 million – 2 million tie! I guess you could say that these are both “perfectly close” results, as verified by their ideal relative differences of 0. But I think most people would regard the 2 million – 2 million tie as way more impressive. And that is indeed what shows up with the adjusted relative difference: the first score is (0 + 1)/(4 + 1) = 0.2, while the second is 1/4,000,001 = 0.00000025, an enormous reduction by a factor of almost a million.
A modest proposal
It was nice that the mathematical justification we gave above recovered (a slight adjustment of) the relative difference statistic that everyone uses anyway. But was that mathematical justification actually convincing, or was I pulling the wool over your eyes?
Well, I wasn’t being deceptive, but there is part of the argument I find unconvincing. It’s where I said that we could consider all 100 results from 99–0 to 0–99 equally likely. Is that a reasonable assumption? If I tossed 99 coins, getting 99 heads and 0 tails would be extraordinarily unlikely – about a one in 600 billion billion billion chance – but getting a 50–49 victory for heads over tails happens quite regularly – about one in every 13 trials.

(This is because there’s only one way for 99 coins to land all heads, but there are many ways for 99 coins to land with a 50/49 split. See the Wikipedia page on the “binomial distribution” for more on this.)
So we should take this into account when calculating our p-value. That is, we should instead calculate the probability of getting a result as close as this or closer with this “coin-tossing” model, rather than the previous “equally likely scores” model. I propose to call the p-value calculated with the coin-tossing model the “Frodsham score” (after the place where the infamous punch occurred that led to the Runcorn byelection – although the “chad score” would also work).
The bad news is that, while there is a formula to calculate the Frodsham score, it’s not a very pleasant one: if the loser gets a votes and the winner b votes, for a total of n = a + b votes, then the Frodsham score is
\[F = 2^{-n} \sum_{k=a}^b \binom{n}{k}\]where that thing in brackets is the “binomial coefficient” (read “n choose k”, and famous from Pascal’s triangle). Pleasantness as a formula aside, though, the Frodsham score is easily calculated on a computer. We get:
| Reform | Labour | adj. rel. diff. | Frodsham |
|---|---|---|---|
| 12,645 | 12,639 | 0.028% | 3.5% |
| Bush | Gore | adj. rel. diff. | Frodsham |
|---|---|---|---|
| 2,912,790 | 2,912,253 | 0.00924% | 17.6% |
To put it another way, under this “coin-tossing” model:
- a result like the Runcorn byelection would happen once every 28 elections;
- a result like Florida 2000 would happen once every 6 elections.
Runcorn wins again! The Florida 2000 result occurs about five times as often.
A final mathematical aside: while the exact formula for the Frodsham score is unpleasant, when the absolute difference d is small and the number of voters n is large, the approximation
\[F \approx \sqrt{\frac{2}{\pi}} \, \frac{d+1}{\sqrt{n}} \approx 0.798 \, \frac{d+1}{\sqrt{n}}\]is pretty accurate:
| Reform | Labour | Frodsham | approx. Frodsham |
|---|---|---|---|
| 12,645 | 12,639 | 3.511% | 3.512% |
| Bush | Gore | Frodsham | approx. Frodsham |
|---|---|---|---|
| 2,912,790 | 2,912,253 | 17.6% | 17.8% |
And, of course, if you just want to compare elections, you can ignore the common factor of 0.798 in the approximation. So my suggestion is this: Don’t take the percentage difference, by taking the difference (or, better, the difference plus 1) divided by n. Divide by the square root of n instead!