A Keeper's Short Passing Ability. An Insignificant Stat That Tells You So Much.

One of the challenges of the recently released Premiership data is trying to marry the figures to on field incidents while at the same time avoiding drawing misleading conclusions because of lack of context or lack of extra detail. Passes and pass completions are one of the most comprehensively covered areas of the data set released by MCFC. Individual passing events are broken down by length, general pitch position and direction, enabling anyone to build  an impressive database relatively quickly.

Below I've summarised the completion rates for all short passes attempted by players during the last campaign. As you would expect, passing is a core footballing skill and the majority of players excel at it. Overall completion rates for all players attempting short passes in all areas of the field exceed 80%

Completion Rates For Short Passes In The Premiership, 2011/12 Season.

Player Position.	Completion Rate % .
Striker.	75.8
Midfielder.	84.7
Defender.	83.9
Keeper.	92.1
All Players.	83.3

However, once we break the completion rates down by position we immediately notice a problem. Seemingly, the most accurate practitioners of the short pass as a group are goal keepers and the most accurate short passer of a football in the EPL last term was MCFC's Joe Hart. With all due respect to Joe, when data throws up broad results that are so obviously at odds with reality, then the analysis of the figures is plainly flawed.

Thomas Sorensen....Stoke's most accurate short passer last term ?

In this case the cause of the anomaly is extremely easy to spot. If a keeper misplaces a short pass he may quickly go from being the last line of defence to the only line of defence. Therefore, keepers are only likely to attempt short passes if they are extremely confident that the pass will be completed. Short passes rolled out to an unmarked defensive colleague are much easier to complete than an attempt to thread a 3 yard pass to a fellow striker in a crowded area.

So a goalkeeper's superior passing completion rate comes about because of the type of short passes they choose to attempt. We can further confirm our suspicions that keepers are making predominately easy short passes by looking at the spread of short passing talent that appears to exist between last year's batch of Premiership keepers.

The easier a task is to complete then the more difficult it becomes to quantify the different levels of skill of those undertaking the task. For example, asking a group of individuals to work out 2+2 won't really help you if you are trying to sort the mathematically gifted from the merely numerate. If we calculate the spread of short passing talent within 2011/12 keepers, we discover that the apparent difference in talent is indeed tighter for keepers compared to the values we see for defenders, midfielders and strikers. In short the keepers are, probably wisely, asking themselves easy questions.

A final additional conclusion can be drawn from this trawl through the passing ability of goalies. After correcting for such variables as number of short passes attempted, we find that teammates who shared duties last year appear to have similar levels of passing ability. Sorensen and Begovic, Jaaskelainen and Bogdan, De Gea and Lindergaard, Given and Guzan, Cerny and Bunn, Reina and Doni, each pair's true short passing ability is within just a few percentage points of each other. This may be coincidental, but it's more likely to also reflect the ability of the relatively constant group of players to whom they are passing. A pass is an interaction between two players and if the receiver is constantly allowing the pass to roll under his boot, his mistake will be reflected in the conversion rates of the passer.

A keeper's passing ability isn't near the top of a scout's must have list, so listing their regressed conversion rates for each keeper based on last year's data isn't likely to interest too many people. However, in piecing together the reason's behind Joe Hart's apparent excellence we have uncovered factors that almost certainly contribute to pass conversion rates for other players in other positions within the team.

Explicit mention of the difficulty of the pass choice is absent from the aggregated list, but it appears to be such a large contributor to conversion rates that it can even elevate goalkeepers as a group ahead of presumable more gifted passing midfielders. It is therefore a hugely important factor. A keeper's choice of the easy short passing option compared to his outfield teammates was easy to spot, but less noticeable is discrepancy in pass difficulty choices made by players who occupy similar positions in a team. The player with the higher completion rate may also be choosing an easier option and that has implications for player evaluation and balancing the risk reward of on field actions.

By looking at an obscure goalkeeping statistic, we've highlighted the need to try to quantify the levels of difficulty involved in individual player's on field attempts, the need to acknowledge the level of talent surrounding him and to analyse using only data derived from players plying their craft in similar playing positions, where the risk reward balance for passes will be similar.

The next post will look at the how we can begin to address these issues and contextualize passing for outfield players using the available MCFC data.

Luck In Sport.

I've never tried my hand at archery except on the Wii and I strongly suspect that competence on the video version doesn't adequately equip you for the real thing. Therefore, should I be lucky enough to be allowed to challenge an Olympic archer it would become rapidly obvious where the talent lay in the one sided competition.

Let's imagine I scored zero bulls from ten attempts and my opponents scored a perfect 10. Our  knowledge of the task being undertaken is enough for us to decide that hitting the bull is a talent and the respective scores should give us a strong indication that the gulf in talent between myself and my opponent is huge. From this limited information we could probably conclude that the Olympian could hit around 99% of such shots because they are yet to miss, although we have only seen 10 trials. I could be generously given a likely future success rate of say 1% purely on the basis that I have a bow in my hand and am competing.

If we repeat the process and the returns are 9 from 10 for the archer and 1 from 10 for me. We can begin to make another informed opinion. The archer is still clearly better than me, but no longer perfect and I have shown enough ability to hit one bull from ten attempts. A reasonable new assessment of the gulf in talent would put the archer at just below his actual 90% strike rate and me just above my figure of 10%. Natural random variation has perhaps favoured the professional and deserted me and I'm actually a little bit better than I've shown and my opponent is really slightly worse than their fine return implies.

We can continue this balancing of random variation and true ability as our respective scores converge, but at some point, say 6 out of 10 and 4 out of 10, we cease to become sure that the discrepancy in scores is still down to a combination of the two factors. It may now be just down to the same forces that allow a fair coin to yield 6 heads from 10 tosses. In short I may now be the equal of my opponent based on talent, but one of us has had a lucky day and the other hasn't.

And if two "coins" can demonstrate variation in "talent" where none exists in limited trials, then so can sports players. It's a point worth remembering the next time you look at raw conversion rates for both teams and individuals and try to pick out "the best".

How Fouls Turn Into Cards.

A largely uneventful Premiership clash between Arsene Wenger's Arsenal and Tony Pulis' Stoke City at the Britannia Stadium on Sunday was partly enlivened by the pregame war of words between the two respective managers. Their antipathy towards each other isn't well disguised and the current spat predictably centred around the disciplinary record of both sides. Pulis mused that Arsenal were hardly strangers to red or yellow cards during Wenger's tenure at the club, while Arsene wondered aloud as to where Stoke had finished in last season's Barclay's Fair Play League.

At first glance Wenger's point was the stronger, as Stoke finished 20th and bottom of the Fair Play League and Arsenal finished 7th.  However, Barclays and the Premier League appear to have opted for a wide definition of Fair Play and red or yellow cards only comprise a minor portion of the total points used to decide the table order. An identical proportion of points are also awarded for "positive play". Points are awarded for adopting an attacking outlook and continuing to press for goals even when a team is already in the lead.

It is rather apt that a competition designed to reward risk taking, even when this approach may endanger wider objectives, such as Premiership survival, should be sponsored by a global bank and unsurprisingly Stoke and other less successful sides suffer particularly in this category. Taking only cards accrued, Stoke would have finished 8th compared to 17th for The Gunners. Few people would immediately recognize attacking intent as an obvious component of fair play, so perhaps we need a more focused method based on fouls and cards.

The recent data dump of player actions from the 2011/12 season has enabled a much more granular approach to be undertaken and while there are obviously flaws in much of the analysis we can begin to explore cards and discipline in much more detail. The data contains every match appearance for every player over last season and fouls and cards are recorded on a game by game basis, although the reason for the caution is omitted. Around 80% of the cards issued in the EPL were for foul play, offences such as dissent to account for the rest. We can therefore delete players from the list who were cautioned without committing a foul, leaving a fairly homogeneous sample that relates cards issued largely to fouls.

I first set a baseline probability for the number of fouls a player needs to give away before a booking becomes  the most likely outcome. Pitch position and recklessness of the contact isn't recorded, but we can use player position as a reasonable proxy for the former. Defenders will be making the majority of their challenges and potential fouls in areas in and around their own box and they will also be more likely to be illegally preventing chances from being created or taken. Therefore it seems reasonable to assume that they will be given less leeway than an habitually "clumsy" striker. Regression analysis was used on last year's individual player data to calculate the chances of each set of defenders, midfielders or strikers ending a match with at least a yellow card to their name given various levels of fouling.

The chart demonstrates the handicap under which defenders have to operate, they are over twice as likely to receive a card for committing the same number of foul challenges as are strikers, the players they are more often challenging for the ball. Strikers only become more likely than not to leave the pitch with a caution when they have conceded 8 or more fouls compared to just 4 for defenders. Midfielders are allowed 5 challenges before a caution becomes odds on and are also treated much less leniently than out and out attackers. A striker who infringes frequently near to the opponents goal rarely seems to be cautioned for persistent fouling, whereas a defender risk this penalty much earlier in the cycle.

To take an obvious example, Bolton's Kevin Davies in 2011/12 committed 1 foul in each of seven games, 2  fouls three times, 3 fouls four times, 4 fouls twice and 5 fouls three times. Using the regression lines for strikers he would have expected to receive three yellow cards last term, which is precisely what happened. Had he been judged as a defender, he could have expected an average of just over seven cautions.

Davies pulls Huth's shirt, Huth kicks Davies, the ball is just a passing bystander.

Few will be surprised that defenders have to be especially careful about testing a referee's patience through persistent fouling and another preconceived notion that we can test involves the general leniency allowed to home players compared to travelling guests. Defenders require fewer indiscretions before they enter carding territory, so I've similarly calculated the likelihood of a booking arising for increasing number of fouls committed by defenders both at home and on the road. The effect is much less pronounced in this case, but it appears that two typical fouls away from home gives you a 1 in three chance of seeing yellow compared to the slightly more lenient sanction of 2 from 7 chances if the offences come in front of their own fans.

How Defenders Who Foul Fare At Home and On The Road.

Every step along the road to transgression, visitors are more at risk. However, we are only dealing with raw counted numbers here. Pitch position for the foul may be more advanced away from home due to a more adventurous approach from the hosts and the referee may, quite rightly judge a foul on the edge of the box more worthy of a caution than two in more benign areas of the pitch. Effectively elevating the overall likelihood of the same number of fouls by a visiting defender resulting in a caution compared to a hometown player. We certainly have evidence for visitors being more harshly treated in terms of foul numbers leading to cards, but that harshness may be fully justified.

These two examples begin to show the depth of analysis that is possible with more granular data. Bolton's apparently lenient treatment from referees last season, where they committed on average, a near league high number of fouls per booking is fully explained by the likely area of the pitch in which the fouls were made. Bolton's forwards were responsible for 34% of their team fouls compared to a league average of just 22%.

Similar broad conclusions can be teased from this extensive data set that may hint at the slight variations in refereeing stance that exists over different match ups and how players of differing styles are dealt with. By isolating games between the Top Four and the rest of the league, there appears to be good evidence that referees are partly protective towards the bigger teams, especially when facing inferior opposition. Players from inferior teams are more likely to be booked after just one foul than their more illustrious opponents in the same game. However, the referees appear to realize that they have taken this stance because the balance then switches with increasing fouls and the players from the Big Four are treated slightly more harshly as they become multiple offenders.

How A Player's Chances Of Being Carded Changes With The Matchup.

No. of Fouls.	Card Probability. All Matchups,    All Players.	Big 4 (vs Rest).	Rest (vs Big 4).	Rest (vs Rest).	Big 4 (vs Big 4).
1	0.167	0.139	0.207	0.161	0.229
2	0.237	0.236	0.255	0.226	0.356
3	0.325	0.369	0.311	0.308	0.508
4	0.426	0.527	0.373	0.403	0.659
5	0.535	0.679	0.439	0.507	0.784

There also appears to be good news for combative players who make many tackles and are involved in lots of one on one duels. Refs appear to appreciate that the risk of fouling increases with increased involvement and players who make a large number of legal challenges are given slightly more leeway when they do foul compared to teammates who make far fewer challenges, but are quickly prone to illegality.

Examples Of How Previous Good Behaviour During A Match Can Help.

Number of Challenges by Player During The Game.	Fouls Committed.	Chance Of Being Booked.
4	1	0.214
34	1	0.107
6	2	0.329
21	2	0.245
4	3	0.495
18	3	0.400
9	4	0.618
21	4	0.539

Bookings, it would seem are more complex than merely taking cards to fouls ratios. Even without field positional data or even considering possible conflicts of cause and effect, we can start to scratch below the surface and begin to see if teams use fouls as a tactical ploy, if they might be aware of which areas of the field and players are less likely to draw a card and if their card count is merited by their foul count.

Where Teams Do Their Fouling.

Team.	Proportion of Fouls By Defenders.	Midfielders.	Strikers.
Arsenal.	34%	56%	10%
Stoke.	42%	30%	27%

We can begin to use all of the above observations to look at the yellow card records of Stoke and Arsenal to get a better comparison than the one provided by the Fair Play League. The table above highlights who committed the fouls for each team and by extension we can conclude where on the pitch they were likely to have occurred and furthermore what their cumulative expected card total would be.

Arsenal appear more likely to disrupt opponents in the midfield region, whereas Stoke operate in the more risky final third of the pitch. Obviously there isn't a clear demarcation line past which a defender cannot make a challenge, but from the available data it makes sense to assume that midfielders make their challenges, on average further up the field than do defenders. If we now for both Stoke and Arsenal isolate the number of fouls committed by each player sorted by position and allowing for such factors as type of opponent and venue we can calculate the expected number of cards each side would receive under the current refereeing climate.

Did Stoke and Arsenal Receive The Cards They Deserved In 2011/12.

Team.	Expected Cards From Fouls.	Actual Cards From Fouls.
Stoke.	54	51
Arsenal.	52	51

In light of the necessary approximations the agreement between expectation and reality is good. Stoke committed 450 fouls compared to 400 for Arsenal, but both teams ended the season with 51 yellows by way of foul. Stoke benefited from a larger proportion of fouls by strikers, these tend to occur much higher up the pitch and officials have become accustomed towards treating such offenders more leniently. The Potters also reduced their card count compared to raw foul numbers by virtue of the larger number of challenges made by the defenders especially, that were entirely fair.

In short the respective records of each team, from raw foul numbers to yellow cards last season were, like Sunday's result, stalemated.