Conversion Percentage Inside the Box. Mind The Gap.

So I used 2011/12 seasonal shot data to look at all goal attempts to see if the rate of scoring was merely the result of random variation around a common mean or if their was likely to be a genuine difference between the best and the worst sides in terms of conversion efficiency.

I split the sample between shots inside the box and shots from everywhere else (including shots and goals scored from you own box.....Tim Howard, take a bow), in an attempt to maintain a decent sample size, but to also smooth out any positional shooting preferences among the teams.

The method sees if the distribution of goals scored by each side, given their relative shot totals, is substantially different from the range you may expect if every side is equally talented at converting similar types of chances.

I deliberately left in penalty kicks for shots from inside the box, because I wanted to see how the conclusions changed as certain types of readily identifiable and unevenly distributed shots were removed from the sample.

So the first run included every goal attempt from inside the box. The distribution of goals scored by the twenty sides did appear to differ markedly from the spread you might expect to see if Manchester City had had 450+ attempts and Stoke had 230+ with every other team contained somewhere between those shooting extremes, but all sides had striking talent that was equally adept at converting the chances that fell to them.

Next, I took out penalties, which tend over time to be given to those that do the most attacking and present a significantly higher chance of scoring that other, open play opportunities from inside the box.

Virtually the same result.

Compared to the sample with penalties, we do edge very slightly closer to a distribution of actual goals in 2011/12 that better resembles a random draw from an equally talented 20 team strike force being presented with varying numbers of opportunities. But we still can very safely say that our actual spread of goals from 2011/12 doesn't resemble a lucky dip with a universal  strike rate. About 2% of teams manged at least 60 goals from the distribution of shots actually attempted by teams during 2011/12 in simulations using a universal, average conversion rate. In reality during 2011/12, three teams out of 20 managed to surpass this target.

So I then took out headers.

Overall, headers present a poorer likelihood of success compared to shots and in 2011/12 headers comprised a heft chunk of the total goalmouth attempts for some teams, (no prizes for guessing Stoke).

With headers culled from the data, the difference between the actual distribution and the range you might expect from one drawn from a group of equally lethal strikes, plummeted to within touching distance of each other.

It is just one season, but once you take out penalties and headers, then the number of goals scored by all other means inside the box, still differs from what might expect to occur by random chance where there is no difference in the finishing talents of each forward line, but the gap is small....Very small.

Here's the regressed conversion rates for shots (with the feet) inside the box for sides from 2011/12 suggested by the above analysis.

EPL Side from 2011/12.	Regressed Conversion Rate for Foot Shots Inside the Box %.
Newcastle.	14.9
Arsenal.	14.8
Chelsea.	14.8
Manchester United.	14.7
Norwich.	14.6
Tottenham.	14.4
Wolves.	14.3
Manchester City.	14.2
Aston Villa.	14.1
QPR.	14.1
Stoke.	14.0
Sunderland.	14.0
Bolton.	14.0
Everton.	13.9
Blackburn.	13.9
Swansea.	13.9
Fulham.	13.8
WBA.	13.8
Wigan.	13.2
Liverpool.	13.1

To put these figures into perspective, the difference in conversion rates between top and bottom, given an average number of shots from inside the box (240) accounts for 4 extra goals and that represents about three league points.

If we ignore Newcastle at the top and Liverpool at the bottom, both of whom broke most statistical models during 2011/12, the actual top five from 2011/12 are to be found in the top seven for converting shots inside the box. And relegated Bolton and Blackburn are at least in the bottom half. So the ranking is fairly consistent with league position in May.

By attempting to produce a reasonably sized, homogeneous sample size, the gap between the degree by which real life conversion rates fall, at first slightly and then precipitously towards a random draw is seen. There's still evidence for a talent divide at the very top, but it is narrowing, throwing the importance of shot volume into the spotlight.

Ten years worth of shot data would be nice to see which side of the line shot conversion rates finally settle on!

Begovic Scores! We Draw!

All of the best photographers appear to have a sixth sense about when a worthwhile picture opportunity is about to arise. It was therefore no surprise that my camera had just been returned to my rucksack when Asmir Begovic (a goalkeeper) scored for Stoke (after 13 seconds) against Southampton at the Britannia Stadium on Saturday.

There were clues available that would have indicated that Mark Hughes had a plan. All teams have a preferred end to attack in the second half and there appears to be a tacit agreement between captains that if the visitor wins the coin toss, he will take the kick off, rather than earn the early game wrath of the home supporters by turning the teams around. Therefore, the undercurrent of discontent that accompanied the sides swapping ends after the coin toss on Saturday, turned to slight bemusement as Southampton lined up to take the kickoff.

Stoke had turned themselves around.

The geography of the Britannia Stadium makes it an ideal site for endurance training. The prevailing wind regularly blows in from Trentham, funnels itself through the two open corners at the south end of the ground and then struggles to exit at the single open corner to the left of the Boothen End, tipping a hat to the statue depicting the three ages of Sir Stan as it carries on towards the city.

Stoke now have a chequered history with near gale force winds. In the distant past it has removed the roof of the Butler Street Stand (along with our best striker to pay for the uninsured infrastructure), but the worst it has managed at the Britannia was the late postponement of a game against WBA. On Saturday it provided Stoke with the opening goal by way of partial payback.

In the subsequent press conference, Mark Hughes acknowledged the deliberate decision to play with the wind during the first half, citing the importance of scoring first, which is encouraging. However, it is (hopefully) unlikely that his keeper was considered the most likely scorer. Hughes' apparent encouragement for his sides to shoot from distance, so vividly demonstrated at QPR, must have some bounds.

Following the kick off, Southampton chose to attack Stoke with a series of intricate passes. This quickly broke down, the ball was rolled back to Begovic, who launched a wind assisted punt goalwards. Both Southampton defenders chose to ignore the golden rule of defending, namely "never, ever let the ball bounce" and Boruc was left embarrassed by a slick bounce on the wet surface.

Begovic is the fifth keeper to score in the Premiership and invariably such goals require additional help or unusual circumstances. Tim Howard's effort against Bolton was a replica of Begovic's goal, but Peter Schmeichel, when playing for Villa opened the goalkeeping Premeiership goal tally from the more advanced position of the opposing penalty area. So modelling the likelihood of a keeper netting is going to be hugely situational.

We may have more luck trying to quantify the quickfire timing of the goal.

Stoke v Southampton, two minutes 13 secs from history being made (not shown).

The chance of a goal being scored increases slowly, but inexorably as time elapses as caution and fitness, gives way to adventure and fatigue. Your chances of seeing a goal during the sixty seconds that comprise the 8th minute is only 80% of your chances of seeing one in the 80th.

However, three "sixty second" intervals are completely atypical compared to this gradual cranking up of goal expectation. The 45th is the second most goal laden "minute" followed by the 90th and the reasons are clear. Injury time extends both minutes well beyond sixty seconds leading to two big spikes. Therefore these increased rates are purely artificially created by traditional timing considerations. The second half starts with the first second of the 46th minute, even if the first half stretched well beyond 45 minutes of actual time.

The barren blip that comes with the first minute, by contrast is entirely real. If you chose any sixty second period in the first ten minutes, you are likely to see around 0.7 to 0.8 percent of the total goals scored during the match, on average. But if you plump for the first 60 seconds of a match, you will be lucky to see much more than half of the typical early minutes goal percentage.

Again, the reasons are fairly plain to see. Every game starts with a kickoff. The ball is about as far from either goal as it can possibly be and all eleven players are positioned between the ball and their goal and this formation of maximum protection for each goalmouth is guaranteed to occur during the first minute in every match. Hence the scoring is not only at its lowest because of the usual ebb of intent and desire to score, it is atypically lower because of the requirement to start the game with a kickoff.

So, pulling all the information together, around 0.5% of goals come in the first minute, the first 13 seconds are likely to see less than a pro rata division of this goal expectation because of the guaranteed safe starting position for the ball. A back of the envelop calculation using these figures and the average scoring expectation over the history of the Premiership gives an average goal expectation for the first 13 seconds of a Premiership match of around 0.0015 of a goal. The chances of scoring twice or more in 13 seconds is impossible, therefore a goal in the opening 13 seconds should, under these informed gu-estimations happen around once every 666 matches.

So, unlikely as Begovic's goal was purely from a timing perspective on that particular Saturday afternoon, we should expect to have seen around a dozen such goals scored before the 14th second has elapsed over the 8,326 game history of the Premiership.

Begovic's strike was the sixth such effort, so maybe the primeval order at kickoff takes slightly longer to descend into chaotic normality than I accounted for or Premiership audiences have just been slightly unlucky.

If you have to miss a minute of a match and you want to reduce your chances of missing a goal, chose the first minute, (although you may be really unlucky a miss club history in the making), but at least on Saturday any tardy spectator got to see a match featuring two keepers whom had both scored a career goal, (although Boruc's strike came from the altogether more likely source of the penalty spot).

Finishing and Hitting the Target in the MLS.

The first attempt I made at looking beyond the commonly available football stats of the day, namely goals, used shot and save data from the MLS. The quality of play may not have quite matched that seen in the Premiership of the day, but the amount of data available far outstripped that that was commonly available to the UK newsgroups that preceded blogging.

Attempting to tease the luck from the talent in the shot saving percentages seen in the likes of Kevin Hartman, Tony Meola, Joe Cannon and Tim Howard was a lot easier than trying to sensibly argue who England's current stopper should be. So a belated h/t to Big Soccer, where around half a dozen stat enthusiasts hung out in the dim distant past.

A recent tweet from the influential Steve Fenn, a must follow at @SoccerStatHunt, reminded me of the excellent work that is being done by the guys at http://americansocceranalysis.wordpress.com/ notably, Harrison Crow (@Harrison_Crow). They are collecting and also sharing shot data in the current MLS. So a major h/t to them, the first attribute is fairly common, but the second is extremely rare and most welcome!

The availability of data is the major bottleneck is blog based analysis. Methodologies are fairly standard, but weight and credence to any conclusions only comes with increased sample size. It is fairly easy to develop a novel methodology, but the limited data can still make you look dumb.

Back in the day, shot attempts and outcome was the limit of the data, but the the volume of the data, stretching over seasons and, in the case of keepers, their longevity, still made analysis possible, if with a slightly wider error bar attached. Increased shot volume, it was hoped would even out issues of shot and chance quality, that did not exist to such as degree in either the controlled pitcher/batter contest in baseball or the more restricted playing area of hockey.

 I don't have an MLS photo. Instead here's Clint Dempsey celebrating Sounders' Interest (and a Goal against Stoke).

Nowadays, the still flawed gold standard from blogging shot analysis is data with x,y co ordinates, but often devoid of even the tiniest hint of defensive pressure, except in the most dedicated of collectors. Which is why the MLS data dump at American Soccer Analysis is so welcome. It improves greatly on shot data of the past by partly bridging the gap to professionally collected and protected data with the subdivision of shots into zones. Usually, slicing and dicing sample size leads to noise and over fitting, but ASA's venture may sacrifice sample size, but greatly increase uniformity of events within those smaller samples.

Applying one of my shooting analysis methods to ASA's improved data was therefore both sensible and a nostalgic treat. Broadly, this method assumes that shot outcome is common to each MLS team and centered around the league average. Any apparent deviation in shot accuracy percentage or conversion (and there is bound to be some) is going to be down to random variation and a talent gap in performing these tasks between sides. Quality of opportunity is hopefully controlled by ASA's use of shooting zones. So if we see a wider range of outcomes in the attempts each side made, compared to a random draw using league averages, we can possibly conclude that random variation isn't the only factor at work in deciding the shooting pecking order.

The sectors used along with the data are all available at ASA's site, so I urge everyone to seek it out there, but for partial clarity the sector descriptions are sector's 1,2,4 and 5 are central to the goal and more distant with increasing number and sector 3 is wide within the area and sector 6 is wide to the flanks.

I have taken shooting data from the site for every game played by every side in 2013 and compared the spread in accuracy (in terms of shots that require a save), conversion rates (goals scored) and the undesirable ability to see shots blocked that was recorded by each side against the type of spread expected from those shot numbers if team talent was universally the same in each sector and variation of outcome was purely luck driven.

Do Sector Outcomes Suggest Factors Other Than Random Variation are at Play in the MLS?

Sector taken from American Soccer Analysis Site.	Does Accuracy Deviate from Random?	Does Conversion Rate Deviate from Random?	Does Avoiding Blocked Shots Deviate from Random?
1	Yes	Yes	Barely.
2	Strongly	V Strongly	Random.
3	Yes	Yes	Random
4	Yes	Random	Yes
5	Random	Random	Random
6	Yes	Random	Random

The results are tabulated above. Using shot data from 2013, there does appear to be some evidence that team conversion rates may show a talent differential when strikers are closest to goal. As attempts move further from goal (in the case of zones 4 and 5) and much wider out to the flanks (in case 6), that differential appears to disappear and outcomes become consistent with the average overall conversion rate for the  MLS. In short, skill may exist inside the box, but outside you're hoping to get lucky....in the MLS at least.

A talent for greater (or lesser) shooting accuracy as measured by an attempt requiring a save appears to survive to greater distances and angles or it may show a tactical approach whereby a side is required to "make the keeper work" in expectation of a follow up rebound....Or everything may be the result of insufficient detail contained in the current, admirable data.

I know very little about the specifics of the current MLS, other than Dallas produce technical adept players and Seattle has the coolest kit, but others may make sense of Philly being the best opportunity corrected finishers in sector 1( closet to the goal) and Portland the most efficient in sector 2.

Random variation is ever present in the data, but recourse to this concept as a catch all when a side over or under performs against the league norm, may be less (or more) than fair to player and coaches alike, especially in the absence of any evidence that the talent gap at the very top level has disappeared completely.

To reiterate here's the link to American Soccer Analysis.