Occasionally the newspapers publish stats based articles
that do not relate to sport, but do serve to highlight some of the dubious assumptions
that can be made from such studies.
In the run up to Christmas, a raft of newspapers, including
the Daily
Telegraph reported that the drink driving capital of Britain was Llandrindod
Wells, a small rural town in mid Wales.
LW had over the last 12 months 1.98 convictions per 1,000
drivers, second to Blackpool with 1.85 such convictions. After establishing the
drink driving hotspot, a couple of reasons were then devised to explain the
results, lack of public transport and a belief that an offender will not be
caught in a rural setting, for example.
However, studies comprising very different sample sizes
inevitably lead to conclusions that may fail to represent the true picture.
Most famously a study decided that small schools are inherently better than
large ones because they appeared in disproportionate numbers at the top of a performance table and is quoted in Daniel Kahneman’s book “Thinking, fast and
slow”.
In short, sometimes samples are too small to come to a
reliable conclusion.
LW has a population of just over 5,000. If the town follows
national trends around 80% of the population will be able to legally hold a
driving licence. So, 1.98 convictions per 1,000 drivers implies that 8 cases of
drink driving were successfully caught and prosecuted in LW over the previous 12
months.
If we imagine that one such case went undetected. Now LW has
a conviction rate of 1.75 per 1,000 and they fall to 4th in the
table. Blackpool is now top and it may seem that seaside towns lead to drink
driving.
If convictions drop to 6, LW fall to the middle of the roll
of shame with entirely unexceptional conviction rates per 1,000 drivers. However, two extra cases added to the actual total catapults the town to
2.5 cases per 1,000, well above the next worst, Blackpool.
So it is possibly the size of LW population that has
contributed to making them a headline in the national press. Blackpool, in
contrast has around 118,000 drivers and the conviction rate is much
less susceptible to large changes occurring in that headline rate because of small numerical changes
in convicted or non-convicted cases. Blackpool has probably prosecuted around 280 drink drivers.
Percentages derived from small sample sizes can bounce
around if the raw number of cases alters by just one or two. Just as small schools can be shown to be the
best, as in the study quoted in Kahneman’s book, they can also quickly become
the worst if just a handful of students produce poor results rather than
excellent ones.
To keep the blog sports orientated, let’s use this dubious method to “prove”
that Uttoxeter, population 12,000, a small town on the correct side of the
Staffordshire/Derbyshire border is a hot bed of swimming world records.
Around 12% of the population are in the age group that would
typically hold a world swimming record. So Uttoxeter has around 1,400 potential
champions. They currently have one actual world record holder, Adam Peaty.
Therefore, Uttoxeter has 0.7 world record swimmers per 1,000
likely candidates. This of course would double if we made the conditions gender
specific, but it is still good enough to give it the best headline rate in the country.
So Uttoxeter can be shown to be the place for swimming excellence, but only by using percentages applied to small
sample sizes which obscure, rather than illuminate the less startling reality
of the situation.
Sadly, it is a flawed conclusion, based on the exploits of a single outstanding swimmer, especially as the town doesn’t currently have a swimming pool!
(Update, we do now).
(Update, we do now).