Pages

Tuesday, 8 November 2011

Scoring the First Goal.

Football is in contrast to many other popular sports low scoring and that in part makes it a tense experience to participate in both as a spectator or a competitor.Therefore an opening goal when viewed within the context of a single game is very valuable to the scoring team.We can do a ball park calculation to demonstrate this importance.A game has on average 2.6 goals and lasts 94 minutes,so the first goal will on average be score after about 35 minutes (it's actually later).If we then assume a typical home side would be expected to score 1.5 goals and their visitors 1.1 we can calculate that the Expected Points for the home side if they score first would be 2.4 points and they would avoid defeat in 92 games out of 100.Even the visitors would return with points over 80 times per hundred games should they draw first blood and they'd average about 2 points per game where they took the lead.

Hajduk Split kick off against Stoke.
Having established that opening goals are hugely influential in individual games our next step is to try to come to a method that will indicate which team is more likely to score first.Probably one of the most famous opening goals was scored by San Marino against England in 2003.Although it didn't materially affect the result it was a surprising enough event to suggest that weaker teams are less likely to score first.We can test this intuitive assumption by collecting a large enough sample of games and record the identity of the first team to score and the pre game goal expectancies for each team.If we do this we find that in games where a first goal was scored the proportion of a team's pregame goal expectancy compared to the total goal expectancy for the game is near enough equal to the proportion of first goals that that team scores.

An example is probably in order.If a team would expect,on average to score 2 goals and concede 0.8 goals in a match or score 71.4% of the goals,it will also score about 71.4% of the opening goals in that matchup where a goal occurs.


Therefore we can say that scoring the first goal is a talent based skill,but the case of San Marino and England also indicates that short term outcomes and insufficiently large sample sizes can lead to less likely streaks appearing.If we imagine for a minute that the EPL is a coin flip league with no home advantage and each team has a 50% chance of scoring first in all 38 games.If we simulate 38 coin tosses and repeat it 20 times for each EPL team we would quickly find a team that appeared to score first much more frequently than 50%.But as we've used the same "coin" for each sequence we couldn't have found a team who specialised in scoring the opening goal,we've just hit an unusual sequence partly through a team's skill and partly through pure chance.Of course the EPL isn't a coin flip league,but it is still prone to short term streaks being mistaken for long term trends.

If we use pre game goals expectations to calculate the proportion of first goals the EPL teams were expected to score over a season and realise that 38 games are likely to produce sequences of games that have been more heavily influenced by good fortune that others,we can start to see which teams may have occupied false finishing positions in the final table.We saw here  that it was likely that Ipswich were lucky to score first as often as they did in their penultimate EPL season and their inflated finishing position of fifth was more of a mid to lower half team who got lucky,than a bona fide 5th place finish.A fact confirmed by their relegation the following season.

A Team's Likelihood of Scoring a Game's Opening Goal Compared to their Actual Record.2008-11.

TEAM. SEASON. % of Games where the Team Scored First. Predicted First Goal Scoring %. % Over
Achievement.
Points per
Game.
Arsenal 2008/09 64 67 -5.3 1.89
Arsenal 2009/10 69 69 0.7 1.97
Arsenal 2010/11 63 67 -6.6 1.79
AVilla 2008/09 49 57 -17.5 1.63
AVilla 2009/10 62 58 6.1 1.68
AVilla 2010/11 56 51 8.2 1.26
Blackburn 2008/09 39 47 -19.3 1.08
Blackburn 2009/10 42 46 -9.8 1.32
Blackburn 2010/11 54 45 17.1 1.13
Bolton 2008/09 44 44 0.3 1.08
Bolton 2009/10 47 44 6.8 1.03
Bolton 2010/11 46 46 -0.1 1.21
Chelsea 2008/09 74 72 2.1 2.18
Chelsea 2009/10 76 74 2.2 2.26
Chelsea 2010/11 53 70 -32.6 1.87
Everton 2008/09 61 53 12.5 1.66
Everton 2009/10 56 56 -0.8 1.61
Everton 2010/11 47 56 -18.6 1.42
Fulham 2008/09 53 46 13.8 1.39
Fulham 2009/10 36 45 -23.8 1.21
Fulham 2010/11 49 48 1.0 1.29
Hull 2008/09 31 42 -37.6 0.92
Hull 2009/10 33 39 -17.1 0.79
Liverpool 2008/09 67 69 -3.4 2.26
Liverpool 2009/10 70 69 1.0 1.66
Liverpool 2010/11 65 62 4.4 1.53
ManUtd 2008/09 83 72 13.1 2.37
ManUtd 2009/10 68 71 -5.1 2.24
ManUtd 2010/11 76 70 8.4 2.11
ManCity 2008/09 58 54 6.7 1.32
ManCity 2009/10 49 62 -27.7 1.76
ManCity 2010/11 74 66 10.2 1.87
Middlesbro' 2008/09 44 45 -2.0 0.84
Newcastle 2008/09 29 46 -56.5 0.89
Newcastle 2010/11 51 48 6.7 1.21
Portsmouth 2008/09 41 48 -16.6 1.08
Portsmouth 2009/10 26 40 -55.4 0.74
Stoke 2008/09 44 38 13.9 1.18
Stoke 2009/10 59 44 25.9 1.24
Stoke 2010/11 38 46 -21.6 1.21
Sunderland 2008/09 50 45 10.0 0.95
Sunderland 2009/10 53 46 12.9 1.16
Sunderland 2010/11 50 47 6.0 1.24
Spurs 2008/09 45 54 -18.8 1.34
Spurs 2009/10 69 63 8.2 1.84
Spurs 2010/11 44 59 -33.7 1.63
WBA 2008/09 29 40 -40.1 0.84
WBA 2010/11 32 46 -45.6 1.24
WHU 2008/09 50 46 8.0 1.34
WHU 2009/10 40 46 -15.1 0.92
WHU 2010/11 39 44 -13.2 0.87
Wigan 2008/09 47 47 -0.2 1.18
Wigan 2009/10 47 45 4.7 0.95
Wigan 2010/11 37 41 -10.4 1.11

As we can see within just 38 games there is considerably variation between a team's most likely rate for opening the scoring and their actually real life figures.Chelsea dropped over 30% below the figure you would have expected for a team of their ability in 2010/11.They weren't quite the side they had been in the previous two seasons,but by failing to score first in just under half of their games,they saw their points per game total fall below 2.They were closer to 4th than they were to 1st.In short Chelsea partly through luck found themselves trailing more often than previously and because "luck" has no obligation to immediately compensate them once the goal had been scored,they almost finished 4th.So far this year they've enjoyed the kind of scoring record more akin to their ability,netting the first goal 70% of the time one has been scored and are gaining 2 points per game,although sample size is an obvious caveat.

The sample size issue is also illustrated if we combine the three season records of the original Big Four,Liverpool,Chelsea,ManUtd and Arsenal to produce a sample equivalent of 12 seasons.If Chelsea were unlucky when it came to awakening the vidiprinter last year,fortune smiled on United in 2008/09 when they pipped Liverpool by 4 points.However,overall the Big Four combined for the last 3 seasons were expected to score 69% of first goals in non goalless games and that's just what happened.

Spurs were possibly fortunate to grab their Champions League spot in 2009/10.Random acts of elevated 1st goal scoring gave them a similar actual first goal percentage to rivals Liverpool despite Spurs arguably being inferior to the Reds.Second city rivals,Man City were probably as good a side as Spurs,but unlike them they underperformed in the 1st goal stakes.Tottenham grabbed the last Champions League spot by 3 points.

If luck plays a part in deciding the title,it can have much more serious consequences around the foot of the table.As Ipswich discovered,if your base talent isn't as high as you think,a few bad or simply neutral bounces can see you relegated from a previous position of apparent strength.Newcastle and WBA have been relegated in seasons where they could have reasonably expected to score first more frequently than they did,they were probably better than their points haul indicated and both maintained enough quality to return to the top flight.The same could well be true at West Ham.

Alarm bells should have been ringing pre season at Blackburn,who scored the opener in 54% of games in 2010/11 compared to an expectation of 45%.There is of course no guarantee that they won't get close to last year's figure this term,but the likelihood is that with little change in squad quality they will trend towards a lower percentage and find themselves at risk of relegation.So far this season they have just over 0.5 points per game and have scored the opener in only 27% of games.Opening scoring percentage correlates very well with points per game and a scoring % of 36 or below over a season puts a team on or below the magic 40 points mark that separates the EPL survivors from the dispossessed.

Graph of Points per Game verses Frequency of which a Team Scores the First Goal.























A team's opening goal record does not define a season,but it can hide frailties or abilities that reveal themselves in subsequent seasons.Ipswich should have toasted their 5th spot,but prepared for the next campaign as though they had finished 15th.When team management comes to decide how much squad strengthening is needed to maintain standards or push on to greater things,a quick look at the previous year's table is rarely sufficient.

Saturday, 5 November 2011

Shot Conversion Rates.

San Marino almost certainly wrote themselves a permanent footnote in the history of the World Cup by scoring a goal after just 8.4 seconds of a World Cup qualifier,they also demonstrated how unusual and unlikely events can and do happen.Europe's third smallest nation with a population that could fit comfortably into the Britannia Stadium didn't accomplish their feat against a fellow minnow,but against England and on neutral soil.Five passes and an assist from Stuart Pearce,what could be simpler.Sadly for the footballing,motor racing enthusiasts the remainder of the game took a more predictable route as England ran out comfortable winners,7-1.Roughly the margin of victory  expected by the pre game ratings.

Overall the 6 goal margin of victory would not have surprised too many people,but few would have predicted a San Marino goal,let alone in the game's opening seconds especially as it was the first time the had ever led in a recognised FIFA international.England,the first 9 seconds excepted had performed in a fairly predictable manner,but Gualtieri's goal had demonstrated that the timing and order of goals can be much more erratic.As Cloughie said "it only takes a second to score a goal".


If that's the case we should therefore expect the timings of goals scored to require a much larger sample size before they begin to approach our pre game expectations.If we hold onto this thought it may help us to explain some of the more unusual patterns seen when we look at a team's ability to turn shots into goals.


In the 2010/1 season a number of teams appeared to be much more efficient a turning away shots at goal into goals compared to their efficiency on home turf.Among them,Arsenal who scored a goal every 9.6 shots at home,but needed just 7 shots on the road and Blackburn whose respective figures were 10.6 and 6.3 shots.In essence,these teams appear to be away specialists.However,football is notoriously littered with specialists of many kinds and in virtually every case the label is an illusion caused predominately by random chance and the road warriors more often than not see their stats gravitate to the league's norm.


Arsenal particularly had been in very advantageous positions during their travels in 2010/11,so I added up all the minutes they had led in a game during their away games,reasoning that whilst they were ahead,not only had they probably been very efficient in taking their chances to gain the lead,but once ahead they could look to play a slightly more conservative game,resulting in less speculative efforts on goal and increasingly as the game progresses clearer cut chances as their opponents push on in search of reducing the deficit.In short,for a large part of their away campaign they had found themselves in a position where their already high shooting efficiency was likely to remain high.And if Gualtieri's record breaker against England was a marker,they could have found themselves in those advantageous positions over a short 19 game stretch at least partly through pure chance.


When I looked at Arsenal's "time in the lead" record at home I found it was equally extreme,but in opposition to the away split.They had led at home for just 33% of the available minutes compared to 45% of the available minutes on the road.If their attacking had been cool,clinical and easier on their travels because of their advantageous away positions and increasingly speculative,desperate and more difficult at home because of the unusually large percentage of the home time they spent either trailing or drawing,then that could partly account for the difference in observed efficiencies.


Time spent in the lead,split by venue for teams who were more efficient at converting chances on the road. EPL Season 2010/11.

TEAM. % of Time
Spent Leading at Home.
*Shot
Conversion Rate at Home.
% of Time
Spent Leading  Away.
Shot
Conversion Rate Away.
Wigan 11.7 0.086 17.5 0.093
Arsenal. 33.0 0.104 45.0 0.141
Spurs. 22.4 0.093 28.3 0.106
Blackpool. 21.4 0.119 25.9 0.130
Blackburn. 29.3 0.094 20.4 0.159

*Shot conversion rates are available from a variety of sources.Some count blocked shots in the total,some don't.Therefore there may be slight variations depending upon the source used.

However,one team does not provide reliable evidence,we need a larger sample size.So I recorded the percentage of minutes a team led both at home and away for the last EPL season and matched these numbers up with their shot conversion rate for each type of venue.As we can see from the table above,five teams appeared to be more efficient on their travels and four of those five teams spent a higher proportion of their away games measured by time leading compared their games at home.


The five leading home shot conversion specialists are listed below and all five spent a larger percentage of their home time leading compared to their away games. 

Time spent in the lead,split by venue for teams who were more efficient at converting chances at home.EPL Season 2010/11.

TEAM. % of Time
Spent Leading at Home.
Shot
Conversion Rate at Home.
% of Time
Spent Leading  Away.
Shot
Conversion Rate Away.
Newcastle. 34.0 0.162 17.3 0.083
Wolves. 33.1 0.144 17.2 0.098
Liverpool. 46.7 0.145 18.3 0.101
Bolton. 27.0 0.122 9.6 0.082
Man Utd. 59.7 0.157 23.4 0.126

If we now plot the shot efficiencies against percentage of time spent leading for all the EPL teams from the 2010/11 season for both home and away games we find that there is a reasonably strong relationship between the two,r squared values are in the region of 0.4 for each graph.


























The same graph as above only this time plotted for away games.




























So there appears little doubt that time spent leading a game is correlated to shot efficiency,but the question remains as to which variable drives the relationship.Do teams lead for long periods away from home because they are particularly good at converting away chances.Or are teams able to maintain a high efficiency on the road partly because random chance has allowed their goals to fall in an advantageous order,enabling the "away specialist" to ply their highly efficient start against teams increasingly more concerned with attack than defence.

I would suggest that random chance is the defining factor.If teams have a particular ability to better convert chances on their travels you would expect that ability to persist across seasons,but it doesn't.Arsenal weren't away conversion specialists in 2009/10,neither were Blackburn or Tottenham,but Wolves,who had been home specialists in 2010/11 were apparently away conversion kings in the previous season.This immediately leads you to suspect the fickle hand of fortune isn't too far away.

You would also expect occasional seasons where away specialists predominate,but year on year larger sample sized whole league figures show the on average superior home sides having a higher efficiency than the visitors.Conditions conspire to make home teams about 0.4 of a goal superior to their visitors and better sides on the day lead over large sample sizes for longer and that appears to confirm the relationship between leading and efficiency.Over large sample sizes the away specialist disappears.

I'm not claiming that random goal scoring timings account for all the variation seen in a team's shot efficiency,but it is certainly a factor.Team ability is another,Blackburn for example may be happy to sit on a draw at say Wigan,but Arsenal would be more likely to press forward possibly leading to reduced shooting efficiency if they were in the same position.

In short,the more desperate your situation is the less efficient you become,just look at the passing stats of NFL teams who are trailing.

Tuesday, 1 November 2011

Poor Results Don't Indicate a Decline in Fortune.

Over the course of the EPL season so far Arsenal,Chelsea,Liverpool and Manchester United have each suffered heavy defeats.Arsenal conceded 8 goals at Old Trafford,although they weren't helped by a red card,a similarly player deficient United conceded 6 at home to neighbours City,9 man Liverpool shipped 4 against Spurs and then Arsenal bought everything full circle by scoring 5 in defeating the last of the original Big Four at Stamford Bridge.

Dire warnings of impending doom has variously followed each drubbing,but are conclusions based on one isolated result or even a more prolonged series of under performing games often extreme and inaccurate ?After all Arsenal,for one appear to have recovered fairly quickly from their fall from pre eminence if their 5-3 win at Chelsea is anything to go by......or perhaps I'm just placing too much importance on last Saturday's result.

Heavy defeats seldom signal a sudden precipitous decline in fortune for teams outside the Big Four,so it's likely that successful,well resourced teams such as Man United would be just as able to overcome an embarrassing,but isolated poor result and be able to carry on much as normal.The margin of defeat suffered by the original Big Four is certainly shocking and headline grabbing,but even these must be placed in their proper context.Scoring is certainly higher in the EPL at the moment,leading to such luminaries as Lee Dixon to concluded that officialdom is trying to banish virtually any type of contact from the game,making defending more difficult and ultimately scoring easier.

Secondly,the damage this season has been inflicted on the top teams by one of their own or in Liverpool's case by a team with aspirations to belong amongst the best.

Lastly,red cards contributed to three of the four defeats.United conceded 5 of the 6 goals scored by City after they had had Evans dismissed,including 3 after the 89th minute,Arsenal allowed 2 more goals after Jenkinson's dismissal and 3 of the 4 scored by Tottenham came against 9 Liverpool players.Therefore,if we wish to gague the performance of the Big Four after a heavy defeat we would probably have to eliminate these three games from the sample.That leaves us with precious few occasions where the top teams have been heavily defeated,so an alternative approach is needed.

Pundits are fond of describing the kinds of run a team is on."Undefeated in the last five games" implying invincibility,while "without a win in five" is the usual pre amble before managerial change is discussed.Therefore to bulk up the sample size I looked back over the seasons to find any run of five consecutive EPL games where one of the original Big Four had failed to record at least one win.As you would expect these are unusual occurrences for the Big four,the chances that they would not win at least a game in a run of five matches on average exceeds 100/1,but they do happen around once every 4 seasons per team.

I used the average success rate for the teams during their poor to measure their form over the 5 games.Success rate is calculated by counting a win as 1.0 success points and a draw as a shared 0.5 a point  for each team.I then calculated the average success rate per game for the Big 4 side for 20 games prior to the start of the run and for 20 games subsequent to the poor run.Obviously,this meant that some of the games post or pre the "slump" were played in different seasons,however I have consistently found that more data even at the expense of recency is a better indicator of a team's future performance than if you restrict observation just to the single season.

Average
Success Rate of Big 4 Prior to Poor Run.
Average
Success Rate During Poor Run.
Average
Success Rate of Big 4 After Poor
Run.
0.76 0.40 0.76

Prior to the teams experiencing a 5 game non winning spree the Big 4 were averaging a success rate of 0.76.During the 5 game slump this fell by almost half to 0.4,but after the apparent slump the success rate returned to the levels of the previous 20 matches.It would appear that a run of unusually bad results can be expected from any team,not just the top ones.It would be inconceivable if this wasn't the case,although you would have to trawl a lot of data to find some of the more unusual cases,but they are almost certainly not an indication of a permanent downturn.If 5 under performing results in a row don't indicate immediate ruin for a team,it would seem premature to also herald a team's decline after one thumping.Team quality changes very slowly over time unless there are clear and obvious reasons why a team has suddenly improved or declined.

The Manager of the Month is often viewed as a curse because poorer results often materialise,but if short term bad scorelines occur naturally with little change in a team's underlying skill levels,the same is likely to be true for unusually good results.So when a mid table manager is rewarded for a good month of results,don't be surprised when his team then slumps (aka returns to it's more usual level of performance).......expect it.