Ipswich Town.A case Study in Relegation from the EPL.

Of the 18 Premiership Manager of the Year Awards,12 have gone to managers who hail from Scotland.Unsurprisingly the bulk of this total is down to Sir Alex Ferguson and his mightily successful Manchester United teams and most people would quickly name Kenny Dalglish (at Blackburn) as another recipient.Between them Dalglish and Ferguson account for 11 of the twelve,Dalglish contributes just one in 1994/95,but the identity of the third and final Scottish EPL titan probably requires some thought.

George Burley is probably best remembered as a cultured full back in Bobby Robson's Ipswich sides of the late 70's and early 80's.He provided half of the fullback pairing with Mick Mills when Ipswich upset Arsenal,1-0 in the 1978 FA Cup final.A game notable for the goal scorer,Roger Osbourne collapsing with exhaustion and having to be replaced immediately after he had scored the winner.He was still a regular in the iconic UEFA cup winning team of 1981,containing such Ipswich legends as Muhren, Thijssen, Butcher, Wark,Gates and Brazil,but injury forced him out of the goal laden two legged final.

He served a brief player manager apprenticeship in his native Scotland before taking over at Ipswich via near neighbours Colchester in 1994 and after a series of near misses in the playoffs he steered the East Anglian side back to the top flight in 2000.

We've seen here that the jump in class from Division One (as it was called in 2000) to the Premiership appeared to be marginally easier for teams to accomplish in the earlier years and there are a few examples where promoted teams not only survived,but they also challenged for honours.Burley's Ipswich were one such team,they missed out on a Champions League spot to Leeds on the last day of the season when they failed to beat an already safe Derby,but their 5th place finish was enough to gain UEFA Cup qualification and win the EPL's Manager of the Year award for Burley.

Hopes were understandably high for the next campaign.Ipswich lacked the star studded lineups that had been a constant feature of Burley's time as a player at Portman Road,but they had finished within four points of the runner's up spot and they had gained almost double the amount of points required to avoid the drop that year.The Premiership positions immediately below the top four places show the least correlation between seasons,the upside is extremely limited and there are usually a glut of similarly talented squads around and below that area of the table who can improve markedly in the next year.On average teams finishing in the region of 6th or 7th drop around 4 places in the immediately following season. But 2001/02 was not just anticlimactic for the Tractor Boys it was to be their last season in the top flight and 2000/01 EPL Manager of the Season,George Burley was sacked within weeks of their return to the lower ranks.

So what went wrong and could such a decline be anticipated or avoided ?

The EPL season comprises 38 games and although the general belief is that luck evens itself out over a season,it isn't so.Teams can and do enjoy extended runs where good fortune seems to smile on them and a run of good fortune is not immediately counterbalanced by an equal run of bad luck.All you can reasonably anticipate is that your streak will be followed by a prolonged run of neither excessively good nor excessively bad luck and eons down the line your streak will be a insignificant peak in a universally dull landscape.Therefore,over seasons and over the 20 teams comprising the EPL there will be years where everything falls butterside up and a team scale to heights where they don't necessarily belong.

To see if this happened to Ipswich in 2000/01 we can use two techniques to see if luck propelled them to the top to the EPL.

Scoring the first goal is a great advantage and it makes the prospect of points very likely.The pre game goals expectation of each team is related to the likelihood that they will be the first scoring team in that match.So we can use Ipswich's record in Division One and increasingly their results record in their EPL season to see if they were opening the scoring at higher than expected and likely unsustainable levels.

In their 2000/01 season over the season on a cumulative game by game basis Ipswich were predicted to score about 48 goals and concede 53 over the season and as such would have opened the scoring in just under half of their EPL games.However,no doubt to the delight of their fans and management they bucked the historical trend for EPL sides and opened the scoring in 23 of the 38 games or 61% of the time.Either Ipswich were (much) better than their recent representative results indicated or they were running their luck.

To see which explanation is more likely we can further use the Expected Points haul Ipswich gained after each   first goal was scored.I've charted the cumulative Expected Points gained by Ipswich in games where first they and then their opponent drew first blood.

Expected Points for Ipswich vs Actual Points where Ipswich opened the scoring in 2000/01.

Number of Games.	Cumulative Expected Points after 1st Goal.	Cumulative Actual Points Total.
23	46.0	45

Expected Points for Ipswich vs Actual Points where Ipswich's Opponents opened the scoring in 2000/01.

Number of Games.	Cumulative Expected Points after 1st Goal.	Cumulative Actual Points Total.
15	9.6	18

The games where Ipswich score first,on average result in the actual points gained virtually equalling the Expected Points from that game position.In games where Ipswich went behind they massively out performed the expected number of points total.

However,this should have given Ipswich cause for concern for the 2001/02 season.They grabbed the lead in the majority of games (unusual in itself),but they performed very much to their pre game rating after the goal had been scored in those games.An Ipswich first goal came,on average after 37 minutes so for 1311 subsequent minutes of the season they performed in a manner not inconsistent with their assumed rating and although they outperformed their assumed rating when trailing,the difference is only attributable to 3 extra wins over 15 games.

Their first season performance was definitely atypical.They scored the first goal much more often than would be expected,they retrieved more points than expected when they trailed,but for 1311 minutes of the season they played as a slightly below average EPL side would play.If as seems likely they had gotten lucky in 2000 or caught teams unaware,they should have anticipated a less extreme set of outcomes the next year and that would mean almost certainly less points.

If we chart the 2001/02 season we see a combination of poor signings,the added burden for an over performing side of European football and a team no longer riding their luck,but instead playing closer to their expected levels.Ratings from the previous season and increasingly from 2001/02 indicated that Ipswich should score the opening goal in 43% of games where at least one goal occurred and they scored it in 41% of games.Their run of opening goals had returned to earth.Their actual points very slightly under performed their Expected Points after a game's opening goal.In short they played to their below average rating and they weren't particularly lucky,resulting in relegation by 4 points.

Expected Points for Ipswich vs Actual Points where Ipswich opened the scoring in 2001/02.

Number of Games.	Cumulative Expected Points after 1st Goal.	Cumulative Actual Points Total.
14	29.5	27

Expected Points for Ipswich vs Actual Points where Ipswich's Opponents opened the scoring in 2001/02.

Number of Games.	Cumulative Expected Points after 1st Goal.	Cumulative Actual Points Total.
20	7.0	5

They were only a slightly worse side in 2001/02 compared to 2000/01 and they hadn't deserved to be a top 5 side in their first year and they were probably slightly unfortunate to be relegated the following season,but that's why they play the season just once and not many times over to try to iron out the short term wrinkles.It's the short term wrinkles that makes sport interesting.In 2001/02,they under performed their pre game rating after an opening goal by 4 points,the margin by which they were relegated.Small margins,indeed.

Many were of the opinion that Burley was fortunate to be awarded the Manager of the Year accolade in 2001,amongst them Gerard Houllier,who thought Gerard Houllier should have won it.......he was probably correct.

Expected Points Graphs for EPL Games,October 29th 2011.

The win probability for each team is tracked on a minute by minute basis throughout each game.At any point in a game a team will have a probability of winning the game outright and an associated probability of drawing the game.By multiplying the probability of winning the game by 3 and the probability of drawing by 1 and adding the results together you will get an expected longterm average for the number of points that team will achieve from that position or Expected Points (EP) for short.


Chelsea 3-5 Arsenal.

Scorers:
1-0,Lampard,14'
1-1,van Persie,36'
2-1,Terry,45'
2-2,Santos,49'
2-3,Walcott,56'
3-3,Mata,81
3-4,van Persie,85'
3-5,van Persie,90'

Manchester City 3-1 Wolves.

Scorers:
1-0,Dzeko,52'
2-0,Kolarov,67'
2-0,Red Card,Kompany,74'
2-0,Penalty awarded,74'
2-1,Hunt(pen),75'
3-1,Johnson,90'

Norwich 3-3 Blackburn.

Scorers:
0-1,Hoilett,45'
1-1,Morison,53'
1-2,Yakubu,62'
1-3,Samba,64'
2-3,Johnson,82'
2-3,Penalty awarded,90+4'
3-3,Holt(pen),90+4'.

Are Championship Sides Coping Better in the EPL?

It's hardly inventing the wheel or the iPad to suggest that goal difference is a very powerful football statistic.The more goals you score as a team and the less you concede,then the greater your chances of winning in the low scoring environment that is football.We've seen here and here that average goal difference is both a very good indicator of past success and because it auto correlates strongly it is also an excellent predictor of future performance.

Goals are the currency of football.

The good early season showing of all three promoted sides has led to the predictable conclusion that the gap between the Championship and the EPL is steadily becoming more easily bridgeable.Both QPR and Norwich are currently in the top half of the table and the playoff  winning Swansea are only three points behind the duo.If the former Championship sides were hunting as a pack they would have amassed 33 points from a combined 27 games and be around seven points from safety with eleven games to play.An impressive performance all round,even when allowing for the natural tendency for over achievement in part of a season to be balanced out by slightly less impressive subsequent results.

Recent events naturally lead to a selective bias in the way people analyse events and information.East Anglians,even Ipswich Town followers will have Norwich's fine recent draw at Anfield at the fore front of their thoughts when discussing the EPL,rather than Ipswich's much more substantial achievement of winning playoff promotion in 1999/00 and following up with a fifth place finish in the EPL the following year.

The natural first step is to evaluate the quality of the sides making the journey between the two leagues and their average goal difference is a good tool.It quantifies a team's worth and is easily comparable to the goal difference they achieve in the higher league and although you won't be able to deduce much about the absolute quality of the team,the change in a team's goal difference will indicate how easily they made the jump up in class.

There have been just over fifty Championship sides promoted to the EPL since 1993,so I split the sample in half and compared the average goal difference per game recorded by teams in their promotion season in the Championship and their figures for their first season in the Premier League.

Average Goal Difference per Game Comparisons between the EPL and Championship. 

Season.	Average Goal Difference per Game in Championship.	Average Goal Difference per Game in EPL.
2003-2011	0.66	-0.57
1993-2002	0.67	-0.41

There is very little to chose between the goal difference per game of either group in their Championship phase,but promoted teams fared on average almost two tenths of a goal better in the initial seasons than they did in the most recent seasons in the highest tier.These numbers imply that on average the Championship sides found the transition to to EPL easier to make in the fledgling years of the Premiership.

To try to confirm this initial impression I then plotted  the individual  regression lines for each batch of games to confirm that there was a reasonable correlation between the Championship season and the EPL season and one or two particularly outstanding promotions weren't skewing the averages.Visually there does appear to be a correlation between seasons and the line of best fit is steeper in the earlier batch than in the later one.This appears to confirm that similar average goal differences in the 90's and early 2000's  translated into better goal differences in the first season in the EPL compared to the later batch.

As an example,the line of best fit in the early seasons of the EPL sees an average seasonal goal difference of 0.6 goals in the Championship translating into an average goal difference of -0.5 in the subsequent EPL season compared to  -0.6 goals in the later seasons.Once again earlier EPL years saw the promoted teams faring slightly better judged by the regression line's predictions.Also the average finishing position for promoted teams in their first year of Premiership soccer was 14th between 1993 and 2002,while from 2003-2011 they finished a place lower in 15th.

The goal difference from a team's promotion season would therefore appear to be a reasonable indicator of their likely performance in the higher league,so I then looked at how robust the relationship is for the different categories of promoted sides.The Champions could be expected to be the strongest contenders for future success as they have most likely proved themselves consistently superior to their fellow promoted sides over a 46 game period and if we use the EPL as a guide the most successful teams are also the most consistent.If you plot the goal difference comparison graphs for the Champions,the second placed sides and the sides promoted via the playoffs we find the correlation between seasons is much the strongest for the Champions and the strength of correlation falls for the second placed sides and is almost non existent for the teams emerging from the playoffs.

The correlation between goal difference gained in winning promotion to the EPL and subsequent performance is strongest for the Champions and weakest for those who had to win their Premiership passports at either Cardiff or Wembley.

It would be naive to believe that points gained in the previous season is the sole factor in determining subsequent points totals,there are mainly contributing factors,the most obvious one being money spent in the pre season transfer window.The shape of the points graph for Championship winners finely illustrates this missing factor.The linear fit for the points is relatively strong,however the best correlation is illustrated in the plot below.

The best fitting curve comparing goal difference pre and post a team's promotion to the EPL.



























Four teams who averaged a goal difference of around 0.8 performed relatively badly once in the EPL.A significant number of sides who averaged a goal difference of just 0.6 goals actually acquitted better when they made the leap forward .If we look at the individual teams involved the four 0.8,yet underperforming teams spent virtually nothing in preparation for their EPL campaign.Nottingham Forest,steeped in a tradition where Peter Taylor bought a million pound talent for peanuts from Long Eaton United,sold more players by worth than they bought in their EPL season(and were promptly relegated),Sunderland spent a net £130,000 (and were promptly relegated),Norwich spent £3 million (ditto) and WBA spent £4 million and were relegated by 3 points.

By contrast the group of teams who performed slightly worse in their promotion year than the yo yo group  spent a net average of £18 million and that outlay enabled them to put in a significantly better EPL performance.Charlton spent £18 million,Middlesbro' £16 million,Sunderland £40 million and even the lowest spenders,Sunderland back in 1996 invested £6 million.The high rolling group also outspent the more successful Championship winners,who appeared to reason that their higher quality squad required a slightly smaller outlay in order to survive their initial foray into the top flight and the graphical representation of thir campaigns confirms that their optimism was well founded.The best of the Championship Champions invested an average of £10 million,half the amount spend by the ambitious,but weakest of the champions.The lack of investment,especially early in the history of the EPL is understandably as the risk of instant relegation,high wage bills and potential bankruptcy was very real,but that threat receded with the introduction in 2006 of parachute payments to the demoted teams.

So for the Champions at least transfer money spent and previous goal difference are both significant indicators of first year EPL performance.It's not as significant for runners up and playoff winners,who are coming to the post season transfer scrum somewhat later and could possibly find themselves buying poorer quality players at unattractive and inflated prices.

A Championship champion's likely EPL goal difference is given by

0.01 * (millions spent) + 0.85 * (average goal difference in winning the Championship) - 1.344.

If we therefore bring the article full circle and use the average figures for our imaginary,composite promotion entity comprising the combined records of QPR,Norwich and Swansea.The teams finished 1st,2nd and 3rd and within 8 points of each other,so for this mild diversion I'm going to treat the teams as Champion class,even if only QPR have the silverware to prove it.We find that they would expect to end with a EPL goal difference of -0.71 given their outlay so far and past goal difference.Currently their combined goal difference measured over their combined 27 games is -0.48,so as a group they are finding the EPL marginally easier than their record and the record of all previous Champions would have expected them to.However,the smart money would be on their individual records falling back to earth come May,although almost certainly not as dramatically as the decline experienced by Hull in their 2008/09 season,when they joint topped the table,were unbeaten in all but one of their first nine games,won at the Emirates and White Hart Lane and yet ultimately escaped relegation on the last day of the season. 

Game of the (Mid) Week,Blackburn v Newcastle.

Blackburn 4-3 Newcastle (AET).

If your idea of fun is lots of goals and a fair prospect of an encore or two then the Carling Cup is the place to be.Usually over 20% of the games go to extra time,if not penalties and therefore the average number of  non shootout goals you'll see is comfortably over 3.The competition's restricted to the 92 EPL and Football League clubs,so there's no non league presence to widen the competitive gap between teams,but it's a lesser competition than the FA Cup so managers do delve deeper into their team's roster of players.With over a fifth of the games extending half an hour past 90 minutes,tiredness can become a factor.It's hard to remember a manager who's been sacked for getting eliminated from the Carling Cup in it's many guises,so substitutions tend to be of an attacking nature.

Obviously,there aren't any points up for grabs in knockout football,so I've plotted success probability as opposed to expected points.Success Probability (SP) is the probability that a team will win a game plus half the probability that a team will draw a game from any chosen point in the game.


Success Probability Graph for Blackburn v Newcastle.

Scorers:
1-0,Rochina,5'
2-0,Yakubu,64'
2-1,Guthrie,90'
2-2,Cabaye,90+5'
3-2,Pedersen,99'
3-3,Lovenkrands,10'
4-3,Givet,120'


I'll leave the gripes and groans about the penalties that were,weren't,should and should not have been given to the sites who specialise in match reports.Suffice to say a game that enters the 90th minute with the home team and slight match favourite leading 2-0 and ends up requiring an extra half hour to decide the outcome is a very rare beast indeed.


9955 times out of 10,000 the homes fans will be savouring a victory within the next couple of minutes,only 44 times will they be cursing a dropped two points or contemplating extra time.Just once will they actually lose there and then.The graph illustrates how Newcastle were fighting to overcome the likely odds for almost all of the regulation 90 minutes and it was after Pardew's triple,attacking substitution that they came closest to over hauling Blackburn in the probability stakes.Any Geordie's who left just before the final whistle and failed to follow the game on the drive home,wouldn't have been surprised to hear that their team lost,but would have been amazed to see it was by the odd goal in seven.


In drama,if not importance the game mirrors Man City's Wembley comeback in the playoff final against Tony Pulis' Gillingham around a decade ago.2-0 down entering injury time the current Premiership leaders repeated Newcastle's feat by scoring two late,late goals to take the match into overtime,the first of Manchester's goals coming just after the Gills' keeper had been named man of the match.

EPL Graphs,October 2011.

The win probability for each team is tracked on a minute by minute basis throughout each game.At any point in a game a team will have a probability of winning the game outright and an associated probability of drawing the game.By multiplying the probability of winning the game by 3 and the probability of drawing by 1 and adding the results together you will get an expected longterm average for the number of points that team will achieve from that position or Expected Points (EP) for short.


Wolves v Swansea and Aston Villa v WBA. 


Two very similar games at kickoff,as the respective home teams would have been expected to emerge victorious in around 50% of these type of matches.But from the 23rd minute onwards the games took very different courses,dramatically illustrating how a low scoring sport can produce twists on turns and make football such a compelling spectacle.


35 minutes into the Wolves Swansea game,the hosts were looking at an average of 0.286 points from an increasingly dire situation while later in the day their near neighbours Aston Villa were comfortably anticipating an average haul of nearly 2.5 points.Come fulltime and Wolves had a point,while Villa had nothing.

Scorers:
0-1,Graham,23'
0-2,Allen,35'
1-2,Doyle,84'
2-2,O'Hara,86'

Scorers:
1-0,Bent,23'
1-0,Red Card+Penalty,Herd,35'
1-0,Penalty miss,Brunt,36'
1-1,Olsson,45'
1-2,Scharner,57'

Red Cards and the Big Four.

The win probability for each team is tracked on a minute by minute basis throughout each game.At any point in a game a team will have a probability of winning the game outright and an associated probability of drawing the game.By multiplying the probability of winning the game by 3 and the probability of drawing by 1 and adding the results together you will get an expected longterm average for the number of points that team will achieve from that position or Expected Points (EP) for short.

Every football fan can remember games or incidents within games that are especially significant for their club.They usually signify that their team has reached a new high in it's development or plummeted to new and possibly permanent depths of ineptitude.Stoke fans would probably put November 1st 2008 into the former category.The initial 45 minutes of their first EPL season where they shipped a goal every 15 minutes was still a raw and recent memory,but they had partially steadied the ship with wins over Villa,Sunderland and an obliging nine man Tottenham.They occupied 15th place in the EPL,the average finishing spot for promoted teams in their first year in the higher grade,but defeats at the hands of Bolton, Middlesbro', Everton, Chelsea, Man City and Portsmouth meant that the visit of third placed Arsenal to the Britannia was a daunting prospect for the hosts.

Pre game expected points values made the Gunners huge favourites.They would expect to return with an average of 2.34 points from their visit north,while Stoke would be lucky to average half a point a game.

However,today was a day when one of the least favoured scenarios materialised.Fuller's 11th minute strike would rarely have been a decisive first goal and Stoke had to maintain that lead until after the half hour before they became the team favoured to clinch more points on average from that particular game situation.By the 70th minute Stoke still held the advantage and would be expected to gain an average of 2 points per game in such circumstances,but they tightened their grip three minutes later when Olofinjana stumbled his way into the box and bundled a trademark Delap throw over the line.Instant cult status for "Seyi".

Scorers;
1-0,Fuller,11'(Delap assist)
2-0,Olofinjana,73'(Delap assist)
Red Card,van Persie,Arsenal,76'
2-1,Clichy,90'

If Arsenal had been unaware of their underdog status from the half hour mark onwards,they certainly knew their likely fate now and three minutes later their predicament worsened even more.Stoke's keeper Thomas Sorensen deliberately dallied over picking up the ball in his box,inviting the loitering van Persie to participate in a game of cat and mouse where the Dutchman was by far the junior partner.Van Persie's half hearted lunge for the ball was never going to arrive before Sorensen claimed it for himself and what the subsequent shoulder charge lacked in violent intent it certainly made up for in stupidity.Sorensen collapsed in a heap,although he was careful to remain in front of his goal line and Stoke's favourite referring pantomime villain,Rob Styles set van Persie packing with a straight red card.The dismissal left Arsenal in a hole too deep for even a team of their attacking flair to emerge from and Clichy's 90th minute strike was merely a consolation.

The view  from the stands confirmed Stoke as a team who could compete on their own turf with the league's very best,dreams of a European Tour were a few seasons down the road,but the Potters were no longer considered to be a team out of their depth.Regarding the dismissal,it was difficult not to conclude that van Persie had reacted to the situation of trailing and most probably losing to a team who should have been dispatched with the minimum of fuss and in a manner Chelsea had achieved on the same ground a month earlier.It was a clear case of red card bourne out of frustration.

It's easy to attach too much importance to individual events,so to test the theory that the best teams see red when they fail to reach the heights they generally attain,I recorded all games involving the Big Four,Liverpool,Arsenal, ManUtd and Chelsea where a red card was shown to their players. I charted the pre game Expected Point Values and the EP for the team in the minute before their player was dismissed.Games between the big four were removed from the sample.

Games against inferior opposition where the big four received a red card.2004-2011.

Teams.	Number of Red Card Games.	Average EP at Kickoff	Average EP just before Red Card.	Average % fall in EP	% of Games where EP fell before Red Card.
Man Utd.	12	2.02	1.54	24	75
Arsenal.	15	2.09	1.73	17	67
Chelsea.	16	2.29	1.98	14	56
Liverpool.	6	1.83	1.01	45	83
Overall.	49	2.11	1.68	20	67

As can be seen the average Expected Points just prior to the red card are below the average EP at kickoff both for the big four as a group and for each individual team.The EP fell on average by 20% and in about 70% of the games it was below it's starting point.Games numbers are relatively small for each team,so conclusions are necessarily tentative.Liverpool appear to be the most bad tempered of the original big four,5 of their 6 red card games came when the Reds were performing worse than expected.Chelsea were the team who appeared least concerned with the current match position before they saw red.

The figures appear to confirm that teams who are playing badly by their expected standards are likely to compound the problem by having players dismissed.The question now becomes is the effect universal or is it more pronounced when those struggling to perform are the best teams in the division,after all they have more to lose than teams with lower pre match expectations and their under performance will be self-evident to their players.The average fall in EP at the time of the dismissal is around 20%,so in the next part of this series I'll take a representative sample of games not involving the big four and see if the red card situations remain the same.

Promoted Teams Play it Safe at Home.

Having watched sub Premiership football for over two decades courtesy of Stoke City,from a fans perspective I can say that the jump in class to the EPL would appear to be a mighty one.Everything appeared to happen much,much quicker,the skill levels naturally were higher and mistakes were usually punished ruthlessly.Even mid table perennials stood head and shoulders above the everyday fare Stoke faced in the Championship.So you can only imagine how the players must have felt.

The cliche goes that there are no easy points in the EPL,but newly promoted teams do have the relative sanctuary of facing their fellow promoted teams four times in their first season.So it would be interesting to know how the promoted teams approach these match ups.It's very likely that fellow newboys are finding life in the EPL similarly tough,the average finishing position for promoted teams in their first EPL season is in the bottom quarter.So the teams involved must view these games as both an opportunity to pick up points and conversely a "must not lose" situation.

I've therefore taken the results of all six games involving the newly promoted sides to the EPL since 2001.The average league position of both the home and away teams when the game was played was 15th,which confirms the struggles these teams faced.

Results of games involving the teams promoted to the EPL 2000-2011.

Average Home Goals.	Average Away Goals.	Home Wins.	Draws.	Away Wins.
1.28	1.10	38%	33%	28%

The first point to make is that home advantage is greatly depressed in these games.This is illustrated in the smaller than usual difference between the average goals scored by the home and away sides.Home advantage is historically worth about 4 tenths of a goal in the EPL,however,in these games it is worth just 0.18 of a goal.On average the league position of the home team when these games were played was 15th,as was the away team's,therefore the average goal difference is solely down to home advantage.Similarly depressed is home wins 45-46% is the typical EPL figure for home teams,but just 38% in this case.Draws are higher than expected,a typical game between two teams hovering around 15th place would end stalemated 27% of the time instead of 33%.

So from the figures it would seem that the home side recognising the importance of these games adopts an over cautious approach resulting in less home wins,more draws and less match goals overall.The away side performs about as expected,they score about as many goals as a typical side of their quality would expect to score against a similar class of opponent and they win as often as expected.....but for the home side at least,losing is not an option to contemplate and potential home wins are turned into draws..

Game of the Weekend. Birmingham v Leicester.

The win probability for each team is tracked on a minute by minute basis throughout each game.At any point in a game a team will have a probability of winning the game outright and an associated probability of drawing the game.By multiplying the probability of winning the game by 3 and the probability of drawing by 1 and adding the results together you will get an expected longterm average for the number of points that team will achieve from that position or Expected Points (EP) for short.

0-0,Referee Salisbury is replaced by Mr Wright,45'
0-0 Penalty awarded to Birmingham,50'
1-0,King(pen),50'
1-0,Red Card for Mills,Leicester,56'
2-0,Wood,84'


Substitutions can have a major impact on match results and so it proved when recently relegated Birmingham
 entertained Sven Goran Eriksson's Leicester City in the televised midday kickoff at St Andrews.

The bright start to the game had matched the weather,but like the unseasonal weather,it was destined to be short lived and the game settled down into a scrappy midfield battle with little quality on show at either ends of the field.The first half was as dull as it's Expected Points graph was featureless,but all that changed within ten minutes of the restart.

As substitutions go this game changing one was unusual.Starting referee Graham Salisbury's calf strain saw fourth official Kevin Wright take the field for the second half and he quickly found himself involved in two massive EP shifting decisions.

Firstly,he awarded Brum a 50th minute penalty following Andy King's tackle on Beausejour.On first viewing in real time it looked a harsh call,but repeated replays confirmed that it was indeed the wright call.

Mr Wright then dismissed Leicester's Mills for a two footed,partially airborne lunge just five minutes after the spot kick was converted by Marlon King.Both decisions were correct,but the travelling supporters predictably sang "1-0 to the referee" until Wood finished them off 7 minutes from time.

The penalty award reduced the Foxes EP from 1.26 to 0.65,King's successful conversion further reduced it to 0.44 and Mills' moment of madness sent it all the way down to 0.099.

The EPL and The Championship Compared.

One very effective way to compare different leagues is to plot the monthly variations for each on the same graph.If we do this for the EPL and the Championship it quickly becomes evident that the leagues are fundamentally very similar.The shapes of the plots are mimic each ther quite closely for the majority of the season,differing only in the monthly values.

The EPL would appear to be a generally more attack orientated league.With the exception of the final month of the season,the EPL is more goal laden and certainly more shot laden.It's only in the truncated month of May that the Championship greatly outscores their superior rival.

The Championship commits less fouls in the first three months,but subsequently both the quantity and the variation matches that of the EPL very closely.By contrast the yellow card totals,although following a very similar track to those awards in the EPL,remain almost universally below the totals for the higher division.This implies that the EPL is officiated more strictly than is the Championship.

Monthly Variation in the EPL.

Monthly variation appears to be a well established effect in the EPL and it's most noticeable in the amount fouls committed per game.The number rises slightly after the first month of the season before showing a general downward trend for the rest of the campaign.There are occasional upward blips,most notably in December,possibly as a result of larger festive crowds.The most interesting question regarding foul counts is whether after an initial rise in number the teams self regulate and become more restrained or whether the competitiveness of the tackling remains broadly consistent and it is the officiating that varies. In tournament soccer referees do tend to be excesively harsh in their interpretation of what is a foul in the earlier exchanges before relaxing their stance as the competition progresses.

The reduction in fouls late in the season is possibly easier to understand.Players are either looking forward to the beach or important international tournaments.There will be games that have a huge bearing on relegation or the title,but these type of games will be outnumbered by other less vital matchups.A large downward spike in the number of red cards coupled with a decrease in fouls and increase in shots in April indicates the games becoming more benign later in the season.

Correlated varibles unsurprisingly tend to follow each other step by step.Goals peak relatively early in the season,level off mid year before peaking again towards the end of the campaign. Shots,shots on target and corners generally follow the trend set by goals.

The really interesting question is why the monthly variation happens.Is it down to coaching tendencies,is some of it driven by the officials or is it weather related?

Do Referees Subconsciously Influence Matches?

The Expected Points Model I use in this blog is a very potent weapon in analysis performance based questions in soccer.All things being equal the cumulative total pre game EP over a reasonably large number of games for individual teams almost always tallies up to be very close to the actual points a team achieves.If there is a major discrepancy between the actual and predicted points totals there is usually a readily discernible reason why.For example if we were to by pure chance select a series of games where the team was destined to have a player dismissed,the cumulative pre game EP for that team would be well above the actual points gained.However,the cause,namely the red cards would be readily apparent.

It is therefore possible to use an Expected Points approach to evaluate any influence a certain referee may
have had on the outcome of a batch of games.However,before I go any further I want to emphasis that because a referee shows up as an outlier in any analysis involving match decisions,it no way implies that there may be deliberate bias involved.People can respond subconsciously in different ways to very similar situations and if the majority react in a different way to you that difference can become apparent.It may present a problem in terms of consistency of decisions,but it doesn't automatically mean that there is deliberate bias at work.

Secondly,random events can converge to make a set of games appear unusual and the result of a common denominator such as the referee,when in fact they have just come about by random chance.I'll deal with these issues in a later post,but for now here's how games controlled by the major EPL ref's from the last seven seasons have panned out.

Recent and major decisions tend to be over emphasised in people's memories,so it's hardly surprising that every set of fans think that every official is out to get their team regardless of venue and regardless of their team's standing in the match in question.But it is an enduring opinion that refs favour either the home teams or the better teams in a game.This is reinforced because in general home sides and favourites will do more attacking,receive more free kicks and penalties and although this can be construed as favouritism from officialdom,it is really just down the better team getting it's just deserts.

These natural advantages enjoyed by home teams is already accounted for in the Expected Points calculation,so we can compare the cumulative pre game EP totals with the actual totals gained by the host and then sort them by referee to see if any patterns begin to emerge.All EPL games from the 2004-05 season until the present were included and any referee who officiated 30 games or less was excluded.The vast majority of the referees appear to officiate in games which on average produce actual cumulative points tallies that are extremely close to the pre game expectations.Mark Halsey for example officiated in 121 games and pre game estimates would have suggested that the home team would gain 198.5 points from those games.They actually gained 198 points.So from these figures we would have to conclude that Halsey was officiating in a way that corresponds very closely to the way officials have refereed in the past.Overall 13 of the 21 referees featured in the table saw their groups of matches produce home points that were within 4% or better of their expected value and the majority have been rewarded by being appointed to take charge of the FA Cup final.

Games officiated by Peter Walton saw the home side's actual points under perform the expected value by nearly 8% (two refs produced higher discrepancies,but their game tallies were each much less than Walton's total of 134 games).Again a caveat must apply.Firstly,these results could have arose through random chance or alternatively Walton may actually be one of the few officials who is able to resist the subconscious influence of the home support and it is his colleagues who are out of step.At the other end of the spectrum Mark Clattenburg is the ref with most games in the survey who saw an elevated points tally for the home teams he officiated.Home teams gained an average of 7% more points than expected when he was in charge.The most prestigious game either Clattenberg or Walton have taken charge of is the Community Shield.

Points accrued by home teams compared to pre game expectations sorted by referee.2005-11.

REFEREE.	PreGame Expected Points for Home Teams.	Actual Home Team Points.	% Difference.	Number of Games.
L Mason.	145.3	157	8.0	86
M Clattenburg.	210.1	224	6.6	124
L Probert.	126.0	134	6.3	72
K Friend.	56.9	60	5.4	33
U Rennie.	81.3	83	2.1	50
C Foy.	247.4	251	1.4	145
P Dowd.	251.5	255	1.4	152
M Dean.	274.1	276	0.7	169
M Halsey.	198.5	198	-0.3	121
M Riley.	178.7	178	-0.4	118
M Atkinson.	256.4	254	-0.9	151
A Marriner.	177.6	175	-1.5	109
H Webb.	294.2	289	-1.8	177
A Wiley.	244.0	239	-2.1	145
R Styles.	186.3	180	-3.4	111
G Poll.	104.6	101	-3.5	61
S Bennett	231.4	222	-4.1	139
P Walton.	231.7	214	-7.6	134
M Jones.	109.0	100	-8.3	60
S Atwell.	69.8	62	-11.2	36

* denotes a cup final referee.

If we repeat the exercise by tracing the points haul for all pregame favourites a similar pattern appears.14 of the listed refs see their games come in at within 4% of the expected points total.Game favourites are not always readily identifiable,for example when a relegation candidate entertains a mid table team.So in the majority of these games there's not a huge amount of subliminal pressure to ensure that the "right" team wins.Games overseen by Phil Dowd have coincided with the likes of Arsenal,Chelsea and ManU failing to fulfil their pre game expectations,while Lee Probert controlled matches has seen the favourites overachieve.

Points accrued by match favourites compared to pre game expectations sorted by referee.2005-11

REFEREE.	Pre Game Expected Points for Favourites.	Actual  Points won by Favourites.	% Difference.	Number of Games.
L Probert.	133.2	149	11.8	72
A Wiley.	278.4	296	6.3	145
C Foy.	274.1	290	5.8	145
R Styles.	213.2	222	4.1	111
K Friend.	60.6	63	4.0	118
M Riley.	191.1	199	4.0	33
M Halsey.	226.1	234	3.5	121
S Bennett.	268.8	276	2.7	139
G Poll.	111.1	113	1.7	61
U Rennie.	93.7	95	1.4	50
M Clattenburg.	235.5	236	0.2	124
D Gallagher.	63.2	63	-0.3	34
P Walton.	259.8	256	-1.5	134
M Dean.	313.6	309	-1.5	169
M Atkinson.	279.1	272	-2.6	151
A Marriner.	212.0	205	-3.3	109
L Mason	163.2	157	-3.8	86
M Jones.	113.3	109	-3.8	60
H Webb.	324.8	310	-4.6	177
S Atwell.	69.5	62	-10.8	36
P Dowd.	287.7	255	-11.4	152

From the previous table is is evident that the interpretation and implementation of the laws in the EPL are very largely consistent with previous seasons.There are very few refereeing performances that deviate greatly from the expected outcome over a large enough batch of games.The fewer the games the more likely it is we will see batches of games that appear atypical and even where outlying officials have controlled large numbers of games the possibility for freak sequences to appear is real and present.

Poor or contentious decisions are often seen as the result of incompetence or malice,but more often they result from a subconscious desire by the officials to please the majority ( in the case of home field teams) or satisfy the consensus opinion (in the case of favourites).Managers often criticise officials often unjustly,but always to gain a possible advantage in future games where the previous,often well publicised vent will subconsciously "encourage"referees to favour their side to atone for previous mistakes.

Judging by the above results EPL referees are largely immune to such pressures or if they do allow their judgement to be swayed it is within the bounds of historical limits and therefore relatively consistent between referees.

How Well Do Team Statistics Persist Over a Season.

As promised here I've posted below the intra seasonal correlation for some of the more commonly found team statistics seen in soccer.For example shots taken by teams in the first half of the season are strongly related to the average number of shots taken in the second half of the year.However,from the previous post we can see shots are not particularly strongly correlated with success rate.Therefore,although we will be able to predict reasonably well the number of shots a team is likely to have in a future game from past performance,that information will not necessarily be the best to use in predicting future success rate.

From the previous post we can see that goal difference is very strongly correlated with success rate and from the table below we can see there is a reasonably strong intra season auto correlation and for that reason most methods for predicting future performance are based on goal difference.

Yellow and Red cards received by teams are very uncorrelated across a season,possibly indicating that teams can greatly influence this aspect of the game.There is after all a very big incentive to improve discipline to prevent suspensions and the reduction of avoidable cards for transgressions such as dissent should be relatively easy to implement......or maybe it's just that all the dirty players are suspended come Christmas.


Table charting the strength of the correlation between individual team statistics for the EPL's first 19 league games and the second 19 games of the season for the last 7 completed seasons.

VARIABLE.	Correlation from 1st Half of the Season to 2nd Half.
Shots by Team.	0.69
Shots on Target by Team.	0.62
Goal Difference.	0.52
Goals Scored by Team.	0.35
Goals Conceded by Team.	0.35
Shots by Opponents.	0.60
Shots on Target by Opponents.	0.50
Fouls Conceded.	0.37
Fouls by Opponents.	0.42
Corners Won by Team.	0.30
Corners Conceded by Team.	0.44
Yellow Cards received by Team.	0.07
Yellow Cards given to Opponents.	0.22
Red Cards received by Team.	0.004
Red Cards given to Opponents	0.07