Soccernomics and Penalty Shootouts.

Just a quick followup to yesterday's post regarding penalty shootouts, especially as it has attracted a large number of views and quite a few emails.

To clarify, the claim that the team taking the first penalty automatically has a 60% chance of winning before a ball is kick is Soccernomics' claim, not mine. So can people please stop emailing to say it's crazy to suggest that the team shooting first has a 60% chance of winning......I know it is

Soccernomics have taken I believe 129 shootouts where the team kicking first has won 78 of the contests (60.4%) and proposed that the team opening the penalty shootout in future contests will enjoy this advantage.

I do not agree with this interpretation of their figures.

Imagine I toss a fair coin ten times and get 6 heads (60%). Could I claim that the coin is actually biased and heads will enjoy a 60% chance of appearing in future tosses? Intuitively the answer is no. I can try to confirm this intuition by either assuming that the coin is a fair one and calculate how likely I am to get an result as extreme or worse than 6 heads in 10 tosses with such a coin. The answer is 75% . That's a pretty big possibility that my 6 heads have arise by chance from a fair coin. Or I can carry on tossing to get more data.

All you can say about a series of trials is how likely or unlikely is the result if we assume the coin is fair or has varying degrees of bias. Even with 129 trials and 60% heads, we can't say that this is a 60% biased coin, we can only make claims based on likelihood and probability.

Below I've also taken option two, namely conduct more trials. I've recorded the results of just under 700 penalty shootouts, some of the data I'd previously collected, but I've updated it over the last day or so. It comprises all World Cup finals,all Euro Championships, the majority of FA Cup ties, Carling Cup ties, Asian cup matches, ACN games, playoff matches, European club ties and Johnson Paint games. Sources are many and varied, ranging from ESPN, to newspaper reports, wiki and fan uploaded Youtube video. I've read so many Brentford JP match reports, I'm halfway to becoming a fan! In many cases it is impossible to decide which team shot first, so many games have been omitted.

Success Rate of Teams Kicking First in Penalty Shootouts 1974-2012. Various Competitions.

Number Of Penalty Shootouts.	Number Won By Team Kicking First.	% Won By Team Kicking First.
689	358	52

There's a 1 in 3 chances that a fair 50:50 shootout will produce 358 winning contests for the team going first in 689 attempts. Statistical convention suggests that you can start to harbour doubts when you find results that have a 1 in 20 or smaller probability of occurring by chance in a fair device.

How ever you look at penalty shootouts, a claim for a 60% future success rate for first shooters isn't supported by the numbers.

On to other matters.................

Shooting First Is No Advantage in a Penalty Shootout.

According to Soccernomics, the most dramatic ten seconds of a penalty shootout occurs when the BBC are back in the studio and ITV are on a commercial break. Namely the coin toss. 60% of the time, the team that takes the first penalty of a shootout goes onto win the contest and the authors have popped up on line throughout the later stages of the Champions League and during the Euros to reinforce this point. This nugget of advanced soccer analytics is regularly restated on blogs and websites and Prozone used an article on penalty shootouts to confirm " in accordance with the findings of Soccernomics, the apparent advantage of the team that takes the first penalty." 


The claim is certainly an eye catching one. Before we can reach a shootout the game usually must finish stalemated after over two hours of playing time, indicating that both teams are closely matched and therefore it seems reasonable to assume that they possess similar penalty taking and saving skills. The premise stated in the Prozone article is that the pressure of constantly playing catch up adversely affects the team going second. This presupposes that professionals feel stress and that the second team in the shootout are always playing catch up. 


They've used data from every World Cup and European Championship since 1998 as well as three key shootouts from this year's Carling Cup final and the Champions League Final and semi final and initially the evidence seems compelling. Two of the three key club shootouts were won by the team shooting first and 14 of the 18 World and European contests were similarly won. 16 out of 21 in total for a 76% success rate, certainly doesn't seem to be the kind of record you would expect from two equally matched "sides".    


If we assume that each team actually had an equal chance of winning the shootout, it's a trivial matter to treat the 21 shootouts as a series of coin tosses and we can calculate how likely it is that a result as extreme as the one presented for the team shooting first by Prozone will occur. That chance is just under 2.7%. Statistical protocol suggests that with a result of this magnitude we can agree that there is some evidence to suggest that a penalty shootout is not a 50/50 proposition between those shooting first and those going second.


However, there are objections to this conclusion.The 1998 cut off is arbitrarily chosen as is the inclusion of the three key games, but we have much more data that we can draw on. The first shootout in the European Championship Finals was the Panenka final of 1976 and the first World Cup Finals one occurred in the 1982 semi final between West Germany and France. We have now extended the sample size to 37 World and Euro shootouts of which 20 were won by the first up side. Opening shooters still win more, but the success rate is now just 54%. The chances of the team shooting first and producing a 20 to 17 record if each shootout was actually a 50/50 proposition is now nearly 75%, so there's absolutely no reason to suspect we are seeing anything other than a fair contest.


So far we can initially conclude that the Prozone sample of 16 first shot winners from 21 trials gave some evidence that the competition wasn't fair, but that result wasn't statistically significant at 5% levels if we assumed that the first team to shoot had an inbuilt advantage of just 53%. Furthermore, in larger sample sizes containing all Euro and World Cup shootouts of which the Prozone sample was just a subset, statistical significance disappeared completely and the success rate for team's going first was highly likely to have been produced by two equally matched teams of penalty takers.


We are now left with Soccernomics' original 60% sample. The number of shootouts contained in their sample is, I believe 129, an impressive number, but still just a subset of all the penalty shootouts that have been played out. Therefore it is no more valid than Prozone's incomplete subset that supports first shooter supremacy or my more comprehensive sample that suggests we are seeing a fair contest. A 60% strike rate would suggest that on 77 or 78 occasions the contest was won by the team going first and as with the Prozone sample this gives us the statistically significant right to claim that the results are extreme enough to be inconsistent with a fair fight. But statistical significant disappears if we give the team going first an enhanced win probability of just 51.5%, a far cry from the claimed 60%.


Just as pertinently, the 129 game sample is probably only a quarter of all shootouts that have been taken in major leagues and competitions and as we have seen subsets can produce results that may disappear in larger ones.


The original claim is headline grabbing and the inclusion of plausible, subjective justifications for an advantage to exist make the idea appear compelling. It is just the kind of conclusion that advanced soccer statistical analysis should be hoping to discover......but unfortunately, it's not backed up by the numbers.

A follow up post can be read here.