Ranking Free Kick Experts. The Pitfalls.

Evaluating player talent has never been more accessible or more prevalent as a result of the raft of individual statistics that are now available. It is therefore important that we fully appreciate the limitations of such analysis and the main areas where confusion and misleading conclusions can arise. The previous season's numbers generated by a transfer target are routinely compared to those of a team's current squad to decide if he will be an upgrade and if so by how much.

I've posted numerous articles aimed at showing the importance of regressing small sample sized chunks of a player's career numbers towards the mean of a group of his peers to better gauge a players real ability levels and provide a better estimation of future performance. What happens over a limited run of games can be largely influenced by luck, with talent taking the minor role. So a player's raw successes and failures should provide just the starting point for an assessment of his true worth. These figures can be improved upon, but even in this new, regressed form they should carry caveats as well.

Below using data from Optapro I've continued the theme of direct free kicks, their effectiveness and the degree to which scoring from such opportunities is skill based. Initially here, I looked at team records for the past four seasons and as teams tend to carry free kick specialists who monopolize such opportunities, it is sensible to look at individual player statistics. Conversion rates from dircet free kicks for the last ten seasons have been combined to provide samples running into the 50's for the majority of the players and into the hundreds for the likes of Ronaldo and Lampard. Also included in the data is the rate at which players hit the target, as well as the rate at which direct free kicks are blocked. A statistics that is sometimes omitted from free kick and open play attempts. 

I initially saw if the observed conversion rates diverge enough from those expected in a purely random process for us to be able to conclude that we are witnessing skilled input from the players. And secondly I've determined the amount of average performance that needs to be included in a team's actual record to improve our estimation of that team's true ability at performing the task we have measured. This analysis assumes that the average difficulty of each chance is broadly similar, it also ignores individual player ageing patterns and any improvement or deteriorating in ability that may come with a player's ability peaking and then declining.

As in the previous post the distribution of success rates among players does indicate that some have more talent at executing a direct free kick than others, but that spread of talent within shooters from the EPL over the last ten seasons is fairly tight. In the table below I've regressed the various conversion rates for every EPL player who has attempted 40 or more shots directly from a free kick since 2002. Scoring from the attempt is obviously the most desirable result, but I've also included the rate at which a player forces an attempted save from the keeper because accuracy can also lead to a valuable second opportunity for a colleague. The ability to avoid the wall with their attempts is included for similar reasons.

Most Proficient Strikers of a Direct Free Kick, EPL 2002-2012.

Player.	Regressed Conversion Rate%.	Conversion Ranking.	Regressed Accuracy Rate%.	Accuracy Ranking.	% Rate of Blocked Attempts.	Blocked Shots Ranking.
Larsson.	8.6	1	44.3	1	27.2	2
Solano.	7.4	2	32.6	6	35.3	16
Henry.	7.2	3	37.3	2	29.8	4
N'Zogbia.	7.2	4	35.2	4	30.1	7
Hasselbaink.	7.1	5	24.9	36	39.9	33
Brunt.	6.7	6	25.8	32	36.3	23
Murphy.	6.7	7	25.7	33	36.5	24
Tevez.	6.6	8	29.8	18	35.4	17
Geremi.	6.4	9	31.9	7	30.4	8
C Jensen.	6.4	10	30.8	14	36.0	19
Ronaldo.	6.4	11	34.0	5	37.9	30
A Young	6.4	12	31.5	9	30.0	6
Giggs.	6.4	13	27.8	25	35.9	18
K Richardson.	6.2	14	26.7	29	30.9	10
Drogba.	6.2	15	31.1	12	30.9	9
R Taylor.	6.0	16	23.3	39	37.2	26
Arteta.	6.0	17	24.6	37	42.8	39
M Taylor	5.9	18	27.0	28	42.0	38
Baines.	5.9	19	24.6	38	37.7	28
Rodallega.	5.8	20	30.1	16	34.7	13
Rochemback.	5.8	21	25.2	34	50.1	40
Bullard.	5.8	22	29.4	20	36.1	21
van Persie.	5.7	23	31.1	11	36.1	20
Pedersen.	5.7	24	29.4	19	39.9	34
Geovanni.	5.7	25	27.5	26	36.7	25
Bentley.	5.7	26	36.8	3	26.8	1
Savage.	5.6	27	31.8	8	34.9	14
Gerrard.	5.6	28	30.4	15	31.7	11
Lampard.	5.5	29	28.1	24	39.4	32
Clement.	5.4	30	21.3	40	41.1	36
L Robert	5.4	31	26.4	30	37.9	29
Nani.	5.4	32	29.2	21	30.0	5
Milner.	5.3	33	28.5	22	39.3	31
Bale.	5.3	34	28.1	23	35.0	15
J Beattie.	5.2	35	30.9	13	36.2	22
Petrov.	5.1	36	31.5	10	34.5	12
Downing.	5.0	37	24.9	35	41.5	37
Barton.	4.9	38	30.1	17	37.2	27
Pennant.	4.8	39	26.1	31	29.0	3
Rooney.	4.7	40	27.2	27	41.0	35

As everyone probably knows, Seb Larsson in his intermittent Premiership career with Arsenal, Birmingham and latterly Sunderland is widely regarded as the best direct free kick specialist. The regressed numbers appear to confirm this, not only is he comfortably ahead of second placed Nobby Solano, he is also the most accurate and second as the least likely to see his shot blocked.

Everyone likes lists, so hopefully the table will be of interest. However, the main point of this post is to try to simulate how easy it is to take one seasons worth of samples and arrive at conclusions that aren't confirmed by future or more extensive datasets.

Shots from direct free kicks for individuals are relatively rare events, especially if the kicking duties are shared out between players. Larsson, who tops the group has double figure attempts in four of his five EPL seasons, the exception being his year at Arsenal when he barely played, but even he doesn't make it into the twenties. Rooney who lies last in the "best of the best" group has only taken double figure attempts since Ronaldo's departure.

So if we restrict our analysis to a single, most recent season for direct free kick analysis we are going to be looking at small sample sizes and that means a lot of luck.Regression towards the mean can avoid drastic over or under inflation of true individual ability by dragging results towards more realistic long term outcomes, but how likely are we to place an inferior practitioner of a particular talent ahead of his superior ?

Broad assumptions have to be made in most new football analysis and if we pretend that Larsson's and Rooney's chances of scoring an individual direct free kick mirrors our estimation of their true conversion average, we can construct a simulation of how often the inferior Rooney, conversion rate 4.7%  would outscore the superior Larsson, conversion rate 8.6% over one (generous) season of 20 attempts each.

In 1,000 simulations, "Rooney" scored more goals directly from free kicks than "Larsson" in just over 20% of the "seasons". So in a rare, but skill based talent in football a player with a conversion rate of under 5% can record a better season long record than one with a conversion rate in excess of 8% about a fifth of the time even when we grant them an inflated number of attempts. Coincidentally, Rooney out converted Larsson in one season out of the four in which they were both playing in the EPL. If the "best" can beat the "worst" a significant number of times over a number of trials, we should be taking great care before we come to solid and forthright opinions about the relative merits of two or more players, even when those conclusions are backed up with real, but raw data.

In short, raw figures are merely the precursor to evaluating player talent, but especially in rare events where talent spreads are narrow we have to acknowledge the real possibility that the rankings may not even fully reflect the true capabilities of the individuals listed.