Wednesday 28 December 2011

Expected Points Graphs for EPL Boxing Day Games,2011.

Chelsea 1 Fulham 1.


Bolton 0 Newcastle 2.


Liverpool 1 Blackburn 1.


Manchester United 5 Wigan 0.

1-0,Red Card,Sammon(Wigan),40'

Sunderland 1 Everton 1.


WBA 0 Manchester City 0.


Stoke City 0 Aston Villa 0.


Home Advantage in the Premiership.

Home field advantage has been an ever present,if declining factor in the top English league since it's inception in 1888.The benefit derived from playing on your home turf back in the 19th century was at times three times as great compared to today's value.Home teams outscored away teams by an average of 0.9 of a goal in the first season of league football and three seasons later the figure had risen to well over 1.This elevated figure from the past no doubt reflects the arduous difficulties involved in travelling relatively large distances to fulfil the fixtures.The amount on average by which home teams currently outscore their visitors is currently of the order of 4 tenths of a goal,although there are fluctuations from season to season.

The easiest way to demonstrate the importance of home field advantage is to see how the chances of two equally matched teams changes if we move the game from a neutral venue to a partisan one.Two typical,equally matched EPL sides at a neutral ground would each have around a 36% chance of winning the game within 90 minutes.However,if we shift the venue,the home side now has around a 46% probability of winning outright compared to the visitors 27% chance.

This is a mighty increase in pre game win probability and it immediately begs the question are some teams able maximize the advantage of playing at home and consistently improve on these already impressive numbers.In short do some teams specialise in winning at home.

Firstly,here's a quick and dirty method to determine home advantage (skip straight to the table if the method becomes too convoluted).Ideally we would need two teams to play a large number of games at a neutral venue,followed by a large number at each other's respective grounds.The change in goal difference from neutral venue to Team A's ground would represent Team A's home advantage and similarly with the change in goal difference from neutral venue to Team B's ground.In reality over one season the closest we can get to this ideal is one team's goal difference against the rest of the league at home and on the road.If we assume that the average home advantage of this team's 19 opponents is going to be reasonably close to the average home advantage for the league as a whole,we can subtract this figure from the spread to give a home advantage,in goals for that particular team in that particular season.

As usual an example to make the situation clearer.Chelsea's average goal difference per game at home last season was +1.37 goals,while on the road it was +0.53 goals.That's a range of 0.84 goals and that figure can be considered to represent the home advantage of every team in the league bar Chelsea plus the average home advantage for Chelsea alone.We know that the average home advantage for all teams last season was 0.42 goals and we can hope that 19 games involving 19 of those 20 teams will average out to close to that figure.So if we subtract 0.42 from 0.84 we get Chelsea's average home advantage for 2010/11 as being equal to 0.47 goals.That's slightly above the league average figure and looking at Chelsea's win/loss home and away split the numbers appear consistent.The success rate for home teams in 2010/11 was 61% greater than that of away teams and Chelsea's home success rate was 63% higher than it's away success rate.

If we do the maths for every team during the 2010/11 season we can get an illustration of how much better or in some cases worse teams played at home compared to their performance on the road assuming that their 38 opponents combined to give performances that were near to the league average.

Home Advantage in the EPL 2010/11.

TEAM. Home Advantage in Goals per Game.
Man Utd. 1.37
Liverpool. 1.26
Newcastle. 1.16
Stoke. 1.10
Blackburn. 0.95
Aston Villa. 0.95
Bolton. 0.89
Wolves. 0.68
Man City. 0.52
Chelsea. 0.47
Birmingham. 0.42
West Brom. 0.42
Tottenham. 0.31
West Ham. 0.31
Everton. 0.16
Blackpool. 0.10
Fulham. 0.05
Sunderland. 0.00
Arsenal. 0.00
Wigan. -0.53

The first thing to notice is that there is a large variation around the average value of 0.42 goals for the league,Manchester United appear to have a massive preference playing at home compared to on the road,while near neighbours Wigan actually performed better away from their home turf.This quite naturally can lead to the impression that better teams manage to muster larger than average home advantages.However,better than average teams such as Spurs and Arsenal appeared to have below average home field advantages last year and lesser teams such as Stoke,Blackburn and Bolton made good use of their home stadium.So it's much more likely that we are merely seeing perfectly natural seasonal variations probably caused by a variety of random events.The values recorded for last season are perfectly acceptable as descriptive statistics for how the likes of Man Utd and Wigan performed at home compared to their away matches during the 2010/11 season,but it provides little in the way of predictive value for future seasons.Wigan for example had a fairly normal home advantage in 2009/10.

To get a better idea of how various team's home and away splits pan out it's necessary to look at more than a season's worth of results.So the next table shows the EPL home advantage for teams who have spent 10 or more seasons in the top flight since 1992.

Home Advantage for EPL Teams 1992-2011.

TEAMS. Home Advantage in Goals per Game.
Newcastle. 0.68
Liverpool. 0.59
Fulham. 0.52
West Ham. 0.49
Man Utd. 0.48
Tottenham. 0.47
Blackburn. 0.46
Man City. 0.41
Sunderland. 0.40
Chelsea. 0.39
Everton. 0.38
Bolton. 0.34
Leeds. 0.34
Middlesbro' 0.34
Arsenal. 0.28
Aston Villa. 0.26

 We now see the seasonal variations disappearing to a great extent and the majority of the teams over a longer timescale have home advantages that are within about a tenth of a goal of the league average of 0.44 of a goal over that time span.There are a few teams that do seem to be slightly more  potent at home compared to away than is typical for EPL teams as a group and also teams such as Arsenal who consistently struggle to make the most of home advantage,but the extremes we saw when home advantage was viewed over a single season are no longer present.

We've seen here that over the relatively short time span of a single season,the partly random nature at which goals arrive during a game can influence more visible global game statistics and it's immediately obvious that Wigan's extremely poor home field advantage in 2010/11 coincides with an abnormal first goal of the game home and away split.On average teams score the game's first goal more often at home than they do away,but through random chance Wigan scored the first goal in just 33% of their home games where a goal as scored.By contrast they notched the first strike in just over 50% of their away games and this abnormal split mirrored their much better form away from home compared to that at the DW Stadium.So it appears that whatever factors allow teams to perform better at home than away on average over the long term can be countered in the short term by other more random factors such as goal scoring times.

To confirm this suspicion,if we plot each EPL team's home field advantage for the 2010/11 season against the differential between those teams scoring the game's first goal at home and on the road,we find that teams with apparently large HFA also score first at home proportionally much more often than the league average.Conversely,teams with very low or even negative HFA scored the first goal away proportionally more often than at home.This leads to the conclusion that short term,abnormal home advantages are likely to be partly the product of the partly random rate at which first goal are scored,they certainly aren't teams specific trends that should be given total credence.In short teams with large home advantages in one season owe that record partly to them getting lucky with how many times they scored first at home compared with on the road.

Seasonal Graph illustrating the increased Home Field Advantage enjoyed by team's who score a higher proportion of  1st goals at home compared to away.
  Having established the likelihood that random factors can skew an individual team's HFA in the short term we should now realise that if we are to make sweeping generalisations about home advantage we must look at much longer term team sample sizes or use the entire season's results.

One area ripe for investigation is how the amount of home field advantage is distributed throughout a game.Do home teams come flying out of the blocks and spurred on by a vocal local support assert their home field advantage early in games and see it gradually decline over the course of the match.This effect has been claimed in various other sports.However,there appears to be little or no evidence that this is the case in football.HFA can be measured in terms of the ratio of goals scored by the home side and if we take the 2010/11 season as being fairly typical we see that the ratio of home goals to away goals was identical to one decimal place last season in the EPL from the first half to the second..

Home Advantage by Half in the EPL 2010/11.

If we extend the study to 10 minute intervals at the expense of sample size,we find that the home/away scoring ratios remain fairly constant with the possible exception of the first 10 minutes.In every 10 minute period from 11 minutes onwards the percentage of home goals scored varies between a high of 61% and a low of 56%,not a huge variation considering we are working with just over 100 scoring events per period.However,the home side scores just 53% of the goals in the first 10 minute period.

Home Advantage per 10 Minute Period,EPL 2010/11.

This may or may not be significant.The first 10 minutes do tend to be slightly abnormal compared to the remaining 80+ minutes in other respects and the effect may be due to a natural caution at the outset displayed by both sides.Alternatively it may indicate that home teams are more passive in their initial approach or that away sides are able,temporarily at least to partly overcome the causes of a home team's natural home advantage.For the moment I'll just present the raw figures,but in later posts we'll see possible causes for a home team's advantage and how they can explain some of the variations we see in the historical data.There are likely to be many contributory factors,we've already seen here that away sides see red more often than they should and we'll also look at how team fare when they move to new stay tuned for "Home Advantage,Part 2 ."

Monday 19 December 2011

EPL and Football League Finishing Positions 2000-2011.

Check out which teams have had a good decade and which teams have slipped down the pyramid.Simply scroll along to find your favourite team.The height of the bar denotes position that team finished in the football league pyramid,so the shorter the bar,the better the team did in that particular season.If you hover the mouse over the coloured bars,you'll be able to see the precise finishing position for your team.

Wolves vs Stoke,Alternative Reality,December 2011.

Wolves 1 Stoke 2.

December hasn't been a great month for referees as barely a week has gone by without at least one major game changing decision being put under the spotlight by an ever critical press.It's a slightly unedifying spectacle to see former referees,especially ones who made high profile mistakes themselves,taking their former colleagues to task for momentary lapses of judgement.However,the incidents do provide great copy for my Expected Points model.

Unusually,Saturday's big beneficiaries were Stoke City.Manager Tony Pulis' innovative tactical ruse of largely dispensing with fullbacks and playing a back four consisting entirely of centre backs is now being copied by the likes of Wenger and at Molineux it was the turn of Jonathan Woodgate to depart centre stage and patrol the flanks.

Woody keeps his eye on the ball.
Despite coping admirably with the new challenge against one of his former teams,Spurs,the previous week,Woodgate looked ill at ease from the first minute as Wolves looked to expose his lack of pace and positional sense.A crude scything lunge after 10 minutes on Matt Jarvis unsurprisingly earned a yellow card and seven minutes later the two players again came together deep into the right hand corner of the Stoke penalty area.Jarvis invited the tackle,cleverly nicked the ball away and went tumbling under another clumsy Woodgate tackle.A penalty award appeared to be just part of Stoke's immediate problems,Woodgate himself appeared resigned to a second yellow card and the Wolves players certainly suggested to referee Anthony Taylor that they expected to be facing 10 Stoke players for the remaining duration.Surprisingly,Taylor opted for a penalty and nothing else and after Hunt had tucked away the spotkick Pulis replaced his converted fullback.

Expected Points if Taylor had Chosen to Dismiss Woodgate Compared to Taylor's Actual Decison.  

Stoke and Wolves were fairly evenly matched just prior to Woodgate's ill judged intervention,but as we've seen previously a combined red card and penalty decision can be a hammer blow to the transgressor.The penalty award made Wolves reasonably strong favourites to grab most points on average and the successful conversion nudged their Expected Points still further into the black.But a red card would have made things much more difficult for Wolves' Staffordshire rivals.Speaking after Huth's freekick and Crouch's far stick header had denied Wolves any return,manager Mick McCarthy was understandably livid that the red card hadn't been shown.His mood was in stark contrast to his delighted post game demeanor  when Wolves had previously claimed an undeserved point at Stoke courtesy of a blatantly offside Jody Craddock header.

Tuesday 13 December 2011

The Continued Stoke/Tottenham Fallout or why Spurs are still in "Sir" Chris Foy's Debt.

Hopefully the irony of lambasting the wrong person for the alleged mistakes of another isn't lost on the Spurs fans who bombarded Sir Chris Hoy's twitter account with complaints about the refereeing performance of his near name sake,Chris Foy.So mistakes everyone.

None of these cyclists is a Premiership referee.
Mistakes or judgement calls that don't tally with someone else's judgement of the incident are always going to happen in football and as I've written previously,luck isn't guaranteed to even itself out over relatively short periods of time.Teams are always going to find themselves in credit or debit to every referee's interpretation of events compared to the consensus "correct" interpretation.Chris Foy has controlled two Stoke/Tottenham games in the EPL,both at the Britannia and both have contained controversial events that have had a major bearing on the game's outcome.

By using my Expected Points model it's a simple matter to calculate the average expected points values for each team prior to each contentious decision and then see by how much those values would have changed had the "correct" call been made.I've already made the calculations for Sunday's game here,so I'll simply tabulate the results.It's probably necessary to point out that Spurs can only have credit for one of the admittedly many poor shouts on Sunday,because as every Dr Who fan knows if you change one significant event,you change the whole timeline.As a consolation though each aggrieved side has been given the alternative future that is most beneficial to them in the final reckoning.

The Costs and Benefits to Spurs and Stoke respectively of Chris Foy's Decisions on Sunday.

Incident. Cost to
Spurs in Average Expected Points.
Benefit to
Stoke in Average Expected Points.
Etherington's Goal.
0.75 0.78
Kaboul's Penalty Appeal.
0.72 0.86
Shawcross Penalty Claim.
1.14 1.37
Adebayor Goal.
0.98 1.29

As we can see the potential Shawcross red card and penalty decision is the biggest game changer for both teams.If the decision had been made it would have cost Stoke an average of 1.37 points and gained Spurs 1.14 expected points.

Chris Foy's only other Stoke/Spurs game came on Stoke's first home game of the 2010/11 season and was sandwiched in the middle of Stoke's run of three straight opening defeats.The game's pivotal incident came just over six minutes from time and is described,with admirable restraint on the BBC website as "Jonathan Walters takes a shot. Blocked by Peter Crouch." In reality Walters' shot crossed the line,was then blocked by Crouch's arm before being cleared.So it was,allowing for the massive confusion surrounding the incident,either a goal or if we allow the officials to err on whether the ball crossed the line,a red card and a penalty.The decision was play on.

How the Stoke Spurs Game would have stood had Chris Foy Allowed Walters' 87' Goal,Aug 2010.

This time it's Spurs who are on the right side of a controversial call so again we can tally up the potential expected points swing.

The Costs and Benefits to Stoke and Spurs respectively of Chris Foy's Decisions on Aug 21st 2010.

Incident. Cost to
Stoke in Average Expected Points.
Benefit to
Spurs in Average Expected Points.
Walters' Goal.
0.95 1.64
Penalty and Red Card Not Awarded Against Crouch. 0.69 1.41

This time Spurs benefit to the tune of 1.64 expected points by Mr Foy decision not to allow the Stoke goal,the worst of the two potential decisions from their viewpoint.Walters' goal standing is the best decision for Stoke and so by incorrectly disallowing the goal Mr Foy cost the Potters 0.95 of an expected point.

If we now go to the final ledger,we see that Spurs are a net 0.5 expected points ahead after we account for a penalty and red card decision that wasn't given on a Sunday night in December 2011 and a disallowed goal in their favour on a Saturday afternoon in August 2010.And Stoke are 0.42 expected points better off because of a Walters goal that was wrongly chalked off and a Shawcross intervention that was leniently dealt with.So ironically both teams in all likelihood have more points than they would have had the calls been correctly made,although Spurs' are now dead and in the record books for last season.The post is not meant to be critical of  Mr Foy as a referee,as this post indicates his record is very similar to those of his fellow officials,he's simply been unlucky in his short term decision making.Everyone makes mistakes,this is the third fifth re draft of this post :-),writing the screenplay to "Back to the Future" must have been a nightmare..

Monday 12 December 2011

Stoke 2 Tottenham 1.The Ultimate Game of What Ifs.

The Britannia Stadium was at it's inhospitable best for the visit of the TV cameras and Tottenham on late Sunday afternoon.The rain was intermittently slanting across the pitch and noise volume was prolonged and loud.The Stoke team were also back to defying convention by winning in their unique and unusual way.See here.The Potters had 40% of shots on target,36% of the possession,but claimed 66% of the goals and 100% of the points,producing another classic "Stoke under Pulis" display.

Stoke 2 Tottenham 1.

2-0,Penalty awarded to Spurs,61'
2-1,Red Card,Kaboul,82'

The bare facts of the game hide a match that was bathed in controversy almost from the start.Red cards,penalties,both given and not given,disallowed goals,goal line handballs,Stoke/Spurs encounters over the EPL years have featured all of these things and they each put in an appearance during Sunday's action packed encounter.

Stoke's Opening Goal with the Score at 0-0.

Crouch,after initially splitting opinion after his big money arrival from Tottenham is proving to be a hit.His continually underestimated ball skills were in evidence after 13 minutes when he cleverly slid the ball across the face of goal for Etherington to bundle home Stoke's opening goal.Unfortunately for the officials he fairly clearly handled the ball just prior to his decisive pass.He also handled at the Brit last season,preventing a Walters equaliser deep into the game when playing for Spurs.Neither official spotted that one either,so there was little sympathy from the Stoke fans as Mr Foy again showed he has an apparent  blindspot when reffing Crouchie.

The goal was enough to propel Stoke's expected points past the diminished Spurs projection.Had the goal not stood,Tottenham would have remained the strong favourites,Stoke's strong start notwithstanding.

Modric's Penalty Award with the Score at 2-0.

Fans are notoriously blinkered when viewing penalties awarded against their own side,so the incessant booing  of Modric following his slightly delayed action tumble following Whelan's clumsy lunge was to be expected.There was enough visual evidence from some of the camera angles to suggest that contact was non existent.However,the intent to foul is often sufficient to cause a player to change direction and lose balance and so Foy's call was at least understandable.Less so,especially given the soft second yellow dished out to Kaboul after fouling Walters in a much less threatening position on 82',was his failure to yellow card Whelan.

 Stoke remained favoured over Spurs from their leading position with just under 30 minutes remaining even allowing for an expected Spurs onslaught as the Londoner's suspicion that the game was retrievable bore fruit.Had the penalty not been awarded and scored,Stoke fans would have been breathing at lot more least for the next couple of minutes.

Kaboul fails to get Spurs a second Penalty with the Score at 2-1.

Mr Foy was probably reluctant to invoke the wrath of the Stoke fans so soon after the Modric award and turned down Kaboul's justified claims for a second spot kick when Shawcross manhandled him as the pair jumped for a cross.Given the amount of contact refs are allowing defenders at the moment,Shawcross would have been right to feel aggrieved if his shirt tug had been punished,but it was a foul.Kaboul was booked for his amusingly mimed protest.

If the decision had gone Spurs' way,Stoke would have clung onto marginal favouritism.As Adebayor had stepped up for the kick,he would have faced a dilemma.Stick with his favoured and already used method of shooting to the keepers left or change sides,simply blast it or allow Modric to take the kick.So there was already enough uncertainty about whether Spurs would actually have scored from the spot.However,the graph has simply factored in the usual 75/25 success rate for penalties.

The 74th minute.

Viewed from row two of the Boothen End the 74th minute didn't appear all that dramatic,but apparently it was.100 yards from the action,most Boothen Enders picked up the linesman's flag before Adebayor turned the ball home,but few saw the two penalty shouts,both against Shawcross and both likely to have resulted in his dismissal.

Defoe Foul/Shawcross Goalline Clearance with the Score at 2-1.

Within seconds Shawcross apparently prevents Defoe from connecting from close range with a shirt tug and then has the presence of mind to cover his undefended goal by clearing Kaboul's cross shot,albeitwith his arm.Had Mr Foy taken a less generous view Stoke's captain would have been off and Adebayor would have been playing mind games with Sorenson again.

The graph refers to the handball,but is equally applicable to the alleged foul on Defoe and demonstrates the absolutely punitive nature of a red card and penalty combined.Stoke would have fallen from being strong favourites to being likely to pick up on average around a point compared to almost 1.5 expected points for Spurs.The graph reflects that the penalty had yet to be missed or scored.

Adebayor's Disallowed Goal with the Score at 2-1.

Marc Wilson's conversion to fullback hasn't be all plain sailing.Despite an excellent season he is occasionally caught ball watching and immediately following the Shawcross incidents he was late tracking out of the box.That was enough to play Adebayor well onside,but with Adebayor and Wilson moving in opposite directions it was only a split second before the Spurs striker appeared to be offside and that optical illusion combined with Shawcross and Sorenson's confident offside appeal was sufficient to get Stoke a lucky verdict.

Had Adebayor's goal stood and the game became tied,Stoke would have found themselves in virtually the same Expected Points situation as they would have been if instead Shawcross had been dismissed and Spurs were preparing to take a potentially game tying penalty.So the two contrasting scenarios are virtually interchangeable and interestingly in my opinion,intuitively neither choice seems "better" for Stoke than the other.

Chelsea Shouldn't Chase Clean Sheets.

Ever since Chelsea's Performance Director mentioned in an interview that "there is a stronger correlation between clean sheets and where you finish than goals scored and where you finish.",there's been a spate of blog posts extolling the virtues of clean sheets.To be fair to the guy he didn't explicitly say that Chelsea were actively using strategies designed to increase their number of clean sheets,but there was certainly an implication that more clean sheets were always a good thing to aspire to.

At first glance the evidence seems compelling.If you keep a clean sheet,you're obviously not going to lose.Score and it's three points and on average you'll pick up upwards of 2.5 points from matches where your opponents remain scoreless.However,by looking at how many points a team gets when they successfully keep their goal intact you are omitting those games where a team goes out with this defensive strategy and it fails.The question a Performance Director should be asking is are the points we get when the strategy works sufficient to compensate for the occasions when we concede a goal (or two) and find ourselves with insufficient time or offensive resources to make an adequate response.You've got to account for the games where you adopt a passive approach,but succumb to a late goal and are unable to reply with a goal or two of your own.

In short,is an overtly defensive strategy the best way for every team ?

If your going to go for clean sheets your games are likely to have less goals,so let's see how a team such as Chelsea fare in first an environment where less goals are scored.Chelsea typically score 70% of the goals scored in their games,so in a low scoring,clean sheet orientated environment where say 2 goals a game on average are scored,they will score 1.4 goals per game and allow 0.6 goals.If we now simulate season after season using these parameters we find that Chelsea,if they adopt a clean sheet strategy will gain on average of 1.97 points per game.If we now simulate seasons where Chelsea still score on average 70% of the goals scored in a game,but this time they adopt a less defensively minded approach and their games now on average see 3 goals per game scored.In an environment where goals,both scored and allowed are more likely and clean sheets are less common a much superior team will see their points per game jump to 2.15.

An average of 2 total goals per game is abnormally low,but it has been chosen to emphasis the effect of a changing goal environment.However,the trend for a superior team to do better in a more goal laden situation is consistent.Over more typical ranges an attacking approach will yield a team of Chelsea's quality around 3 points a season more than if they opt to try to prevent their opponents from scoring.

Chelsea have the wealth and prestige to attract the best attacking talent available,therefore they would be better served forgetting about trying to increase their quota of clean sheets and instead take a more attacking approach and challenge their opponents to keep up.It'll get them more points in the long run and they should leave overtly defensive strategies to teams with less attacking resources.

Footnote.Headline from Daily Mail following Chelsea conceding a late equaliser in a 1-1 draw at Wigan."Chelsea paid the price for negative play, admits AVB after dropping points at Wigan."

Read more:

Friday 9 December 2011

Trailing Teams Show More Attacking Intent in the EPL.

 Ask any casual football fan to describe the atmosphere at the Britannia Stadium,home of Stoke City and the likely impression you'll get is one of a constant cacophony of noise,capable of intimidating even the most hardened of visitors.This view is repeated and reinforced on Match of the Day whenever Stoke turn over a bigger club (usually Arsenal) at home.The Brit is not a place opponents enjoy visiting,but for more regular visitors to Stoke the truth is rather more complex.True there have been occasions when the Brit has literally rocked,a 1-0 win over Man City achieved with ten men for the majority of the match springs to mind as does any visit from City's neighbours United who are still considered,slightly perversely by many older Stoke fans as Stoke's natural rivals.But as with any natural organic process the Britannia crowd ebbs and flows and the decibel levels can sometimes barely register for considerable portions of a match.

The Britannia Stadium,an inhospitable place for opponents....on average.
A much more reliable constant is the physical experience of watching football during the winter months at Stoke.Perched on an exposed outcrop overlooking the highrise concrete of Stoke itself to the north and the more pastoral greenery of Trentham to the west,the ground is landmarked by a massive incinerator chimney,thoughtful positioned upwind to the prevailing gales.Two of the ground's diagonal corners have yet to be filled in resulting in the wind and rain funnelling itself through the ground virtually from November to March.Watching Stoke at home is a very cold experience.

However,despite the freezing night time temperatures few fans were feeling the cold as the left the ground after a 1-1 draw with Dynamo Kyiv.The Stoke fans had a warm glow that came from sealing post Christmas Europa League knockout football and the Ukrainian visitors who,used to much more severe weather had dispensed with a fair proportion of their clothing pre game.Combined with last season's FA Cup final appearance,it was a fine achievement for a team who in the nineties had vied briefly for the title of worst FA Cup team of all time.But the evening itself contained another paradox.

Pre kickoff,the game had shaken down into two possible outcomes.A Kyiv win would likely see them progress at Stoke's expense,while any other result would see the Potters progress.For around 70% of the first half the game had been all square and Stoke had been the team to progress,yet despite this comfortable state of affairs,the Potters had played abysmally.It was only when Kyiv scored and Stoke faced elimination that the home side started to play rather well,culminating in a late equaliser.Just like the famous Stoke atmosphere,the overall outcome was the sum of a couple of diverse extremes.

Rewind to Stoke's first season back in the EPL.Arsenal had been dispatched at the Britannia and Liverpool had escaped with a draw,but the other two of the big four would prove to be a harder nut to crack.A resilient rearguard action in the face of a largely ineffectual Chelsea side at Stamford Bridge was rewarded with a scrappy Delap goal,but having held the London side at bay for an hour the Potters then succumbed to two late goals in the 30 remaining minutes.

Taken in isolation these events are mere anecdotes,but they do imply that there is perhaps some mileage in the widely held belief that teams(and fans for that matter) are able to tailor their effort to suit the game situation.If you wander in on any televised game,mid match,the two things you immediately want to know is who's playing and who's winning (if indeed anyone is).The first question helps you decide the likely winner,the second the likely course the remainder of the game will take.So does the data point to teams being more likely to score when they trail than they were if for instance they were ahead.There certainly seems to be an instantaneous added urgency to a team's efforts once they draw to within a goal of their opponents,witness the common sight of the scorer rushing into the net to retrieve the ball and carry it back to the halfway line.

To help answer this question we first need a baseline to describe what happens to a team's scoring abilities on average.We saw here that the chances of a team scoring the first goal in a game is directly proportional to the amount of goals that team can expect to score on average in that match up.The good news is that what holds true for the first goal is also on average true for any subsequent goals.In addition the rate at which both teams scoring abilities decays with time throughout a game maintains the expected scoring ratio for both teams.In short if you've a 60% chance of scoring the first goal in a game,then you've a 60% chance of scoring the 2nd or 3rd goal.....on average.

However,if we look at the numbers for the EPL over the last half dozens seasons we see that there is a consistent trend for the league teams underscoring the amount of expected goals when a team leads and overscoring their expected amount when they trail.I'll illustrate this with examples from the last 5 seasons using data from two polar and opposites both in talent and alphabetically from the EPL,namely Arsenal and Wigan.

Wigan have consistently struggled over the last couple of seasons,they currently lie bottom and escaped relegation with a last day,78th minute winner at Stoke.So unsurprisingly,on average they aren't usually the favoured team in a match up and by my estimations they were expected to score around 38% of the goals in games over the last five seasons where they trailed at any stage by exactly one goal.Therefore,they'll have around a 38% chance on average of scoring the next goal in those games if the situation of Wigan trailing has no effect on their ability/desire/effort made to score next.However,as the table below shows,when trailing by a goal,Wigan mimicked the league trend and managed to haul themselves up to a scoring rate of 43%.

 Wigan's Scoring Rate when trailing by 1 goal compared to their seasonal average.2007-2011.

Seasons. No.of Times
Wigan Score Next when Trailing by 1
No. of
Times Opponents Score Next when Wigan Trail by 1.
Wigan's Average Pre
Game Goals Expectancy.
Opponent's Average
Pre Game Goals Expectancy.
2007-2011 32 42 1.01 1.67
% of
 Next Goals Scored/Expected to be Scored.
43% 57% 38% 62%

Wigan's Scoring Rate when Leading by 1 goal compared to their seasonal average.2007-2011.

Seasons. No.of Times
Wigan Score Next when Winning by 1
No. of
Times Opponents Score Next when Wigan Lead by 1.
Wigan's Average Pre
Game Goals Expectancy.
Opponent's Average
Pre Game Goals Expectancy.
2007-2011 18 34 1.12 1.44
% of
 Next Goals Scored/Expected to be Scored.
35% 65% 44% 56%

By contrast the reverse is seen when Wigan led by a goal.This time in games where Wigan took a lead of a goal they were expected to score 44% of the next goals,(they generally lead against poorer teams than they trailed to),but this time they scored just 35% of the next strikes.Or to put things another way,teams trailing by a goal to Wigan were more likely to score next than their per game seasonal average expectation would have suggested they were.

Wigan,more likely to be seen on the defensive if they lead the game?

  The case of Arsenal is identical,over the same period they would have been expected under scoreline neutral conditions to score 64% of the next goals in games where they at some time trailed by a goal,but they actually scored 69% of the time.And when they led their ability to score next dropped from an expected seasonal average for games of that type from 72% to an actual 70%.

In short teams who lead become slightly less likely to score next and trailing teams become slightly more likely.Over seasons and sometimes even within individual games this effect tends to cancel out,but within the context of a single game the effect can be a significant component that goes towards explaining the dynamics of the match.The reasons for this universal phenomena are likely to be much more complex.Teams are likely to be pacing their effort to stretch for the full ninety minutes,even in an era of increased fitness and three substitutions,so an immediate,reckless all out assault shouldn't be the expected response from a team falling behind.More common,time permitting,will be a gradual cranking up and re balancing of attacking intent and defensive risk.Similarly,a one goal lead can make a more cautious approach very enticing.

At least if your team is losing you can expect to see them showcasing their attacking abilities to best effect.

Monday 5 December 2011

Expected Points Graphs for EPL Games,December 4th & 5th 2011.

At any point in a game a team will have a probability of winning the game outright and an associated probability of drawing the game.By multiplying the probability of winning the game by 3 and the probability of drawing by 1 and adding the results together you will get an expected longterm average for the number of points that team will achieve from that position or Expected Points (EP) for short.

Everton 0 Stoke 1.


Fulham 1 Liverpool 0.

0-0,Red Card,Spearing(Liverpool),72'

Wolves 2 Sunderland 1 or 26 seconds that Changed a Game.

When a manager's reign reaches the end game he's very much at the mercy of short term luck.Good fortune is chalked down as just that,while the soon to be departing incumbent is held personally responsible for any bad luck that happens along the way.If Brown and Westwood hadn't engaged in a suicidal game of pass the parcel in injury time against Wigan ,Steve Bruce would likely have managed to hang on for a few more weeks at least.As things turned out as Di Santo was walking the ball into the net a week ago,Bruce was as good as walking through the gates of the Stadium of Light for the last time as a manager.A point against Wigan instead of an injury time loss,followed by a couple of back to back wins and Bruce could still have been in a job come May.

For newly appointed managers the scenario is completely different.Much like a President in his first 100 days in office he's immune from any association with even the most bizarre of bad fortune and he gets all the credit when good things happen.Martin O'Neill probably isn't any better a manager than Steve Bruce,but watching from the stands at Molineux a day after his appointment as Sunderland manager he saw his new side take a lead early in the second half and he was about to share in a stroke of good fortune.Wolves' manager Mick McCarthy had gambled on returning Jody Craddock to central defence and the veteran defender had produced a typically solid performance.However,his lunge as the onrushing Seb Larsson darted into the box on 72 minutes was ill judged.Craddock quickly withdrew his out stretched leg,but not before Larsson had taken advantage of the apparently unfolding foul and thrown himself to the turf to earn an undeserved penalty kick.Sunderland,already 1-0 to the good were strong favourites before the decision,they would average just under 2.4 points from their pre penalty position and caught in the twilight zone between award and kick their expected points jumped by an extra 3 ticks.

Wolves 2 Sunderland 1.

0-1,Penalty awarded to Sunderland,72'
0-1,Penalty missed by Sunderland,73'

A successfully converted penalty would have made Sunderland massive favourites to take all three points and possibly heap terminal pressure on Mick McCarthy's tenure at Wolves.But the vagaries of short term chance were about to spectacularly flip flop.Hennessey easily claimed Larsson's weak penalty,thumped the ball downfield and 26 seconds later,Fletcher's powerful header skidded high into the net.Wolves had gone from nearly down and out to marginal favourites in just 26 seconds and then eight minutes later Fletcher's second strike claimed an important victory for the old gold shirts.

Expected Points Graphs for EPL Games,December 3rd 2011.

At any point in a game a team will have a probability of winning the game outright and an associated probability of drawing the game.By multiplying the probability of winning the game by 3 and the probability of drawing by 1 and adding the results together you will get an expected longterm average for the number of points that team will achieve from that position or Expected Points (EP) for short.

Aston Villa 0 Manchester United 1.


Manchester City 5 Norwich 1.


Blackburn 4 Swansea 2.

Red Card,Allen(Swansea),81'

QPR 1 WBA 1. 


Tottenham 3 Bolton 0.

Red Card,Cahill(Bolton),18'

Newcastle 0 Chelsea 3.

0-0,Penalty awarded to Chelsea,13'
0-0,Penalty Missed,Lampard,14'

Wigan 0 Arsenal 4.

0-4,van Persie,79'

Sunday 4 December 2011

Stoke v Dynamo Kyiv.

One of the challenges when trying to make sense of the course a soccer game will take is to account for the changing dynamics that occur during the match.Unlike more established statistically investigated sports such as baseball and American Football,soccer has a partial dearth of numbers to crunch.Therefore,it's a useful exercise to look at the methods used to model more data rich sports and see if comparisons can be made with situations that arise in soccer.

The NFL has a much more structured game dynamic compared to the more fluid situations that exist in soccer.Every NFL play is recorded in the play by play data.Down,distance to go,field position,time remaining and score is available for each game played over the entire duration of the modern NFL,so it's a fairly simple matter to build up an extensive database for most of the available field position combinations.For example,the most common position NFL teams have found themselves in over the seasons is a first and ten from their own 20 yard line (it's the re start position from a touchback).The average outcome from a team starting a drive from their own twenty yard line nets them just over three tenths of a point,they've got about a 70% chance of getting another set of downs and around a quarter of the time they'll score points,either through a touchdown or a field goal.That's an impressive set of conclusions and undoubtedly of interest.However the outcomes are the result of a whole range of diverse matchups from poor defenses taking on elite offenses,to the best verses the best and any combination in between.In short the average outcome is not team specific.You can use this type of data to say what happens on average,but that average is likely to underestimate by some distance the likely outcome when a New England team at their very best takes on a Tampa Bay side vying for the number one draft pick.

Therefore if we use purely historical data as a way of predicting the likely course a sport of any kind will take we are almost certainly going to present a broad representation that inevitably lacks detail.So the first thing we can learn from looking at attempts to model other sporting contest is the importance of the need to account for the differences in overall team strength that exists between the competing teams.Knowing that all NFL teams who found themselves facing a first and ten from their own 20 averaged one touchdown in ever seven such drives tells you little about the number of touchdowns a fully strength Indianapolis side could expect to gain against the worse pass defense in the league.Similarly knowing that EPL sides who lead by a goal at half time won in 70% of games doesn't help you too much when deciding how Wigan would fare if they were fortunate enough to find themselves in that situation at Old Trafford.

Lesson one,match ups matter.

The second lesson we can learn from looking at attempts to model the NFL requires a little more understanding of the intricacies of American Football.The NFL is currently a passing league.Rule changes to make the game more attractive to TV audiences have made defending the pass much more difficult and the average gain made by passing the ball is around 50% greater than the gain to be made by running it.This discrepancy more than makes up for the increased risk of turning the football over to you opponent by going aerial.It also means that there is a reasonably well defined dynamic to the game.Scoring is more likely when you throw the ball,but the risks that go with this approach in terms of incompletions and interceptions also increase,so when a team trails they adopt a strategy of passing more.Alternatively,if a team is protecting a lead they take a more cautious approach by running more frequently than they do if they are trailing.They are less likely to score as freely as they would have done if they'd thrown the ball,but running as well as keeping the clock ticking and hastening the end of the game,also reduces the risk of turning the ball over.In short,teams are more adventurous when behind and more cautious when in front.This shift in strategy is readily observed in the play by play data,but because no comparable information exists in soccer and because of the relatively low scoring nature of the game,it's difficult to observe a similar tactical shift in soccer.

The most potent reason why game strategy is less extreme in soccer than other sports is the elevated likelihood that the game will currently be tied or will end up being tied.Scoring events in the NFL are more frequent compared to soccer,an average game of gridiron has 10 scoring events,while soccer has less than 3.Also a goal in soccer is always worth the same one goal,but a score in the NFL can add 2,3,6,7 or 8 points to the scoreboard.An NFL team is much more likely to be either behind or ahead and that game state will  significantly dictate how adventurous or otherwise their play becomes.By contrast soccer teams are frequently tied for significant periods of a match and that can be a game position that both teams can find find themselves reasonably happy with.We may intuitively guess that one team is putting more effort into scoring than they previously had done,but we may simply be seeing the perfectly normal ebb and flow of a game.

If we want to see whether teams adjust game strategy and become more adventurous when behind and more cautious and defensive when in front we really need a game where the draw is of no value to one of sides and then we should expect to see a polarisation of alternative strategies being played out for the entire 90 minutes dependent upon the current scoreline.The trends will be easier to spot.Which brings us neatly to Stoke's Thursday night's Europa League game five encounter with Dynamo Kyiv.A draw or win guaranteed Stoke qualification for the knockout stages,but Kyiv almost certainly needed a win to keep qualification in their own hands.

Stoke City 1 Dynamo Kyiv 1.


Stoke with homefield advantage were the better team pre game,but they sent out a team that was more
equipped to defend by including two holding midfielders and they also included both of their longthrow experts at the expense of attacking width.Pre game radio quotes that Stoke would not play for a draw also alerted seasoned Pulis watchers to the likelihood that that was exactly what they would do.

For 30 minutes Stoke were happy to sit deep and hit Kyiv on the break and the Ukrainian's did most of the pressing,finally opening the scoring with a deflected shot.They briefly threatened a second killer goal,but as the game wore on they were happy to absorb pressure from a now more attack minded home side.Stoke's fullbacks pushed on,they withdrew one of the holding midfielders to be replaced by an attacker and Kyiv withdrew their star attacking turn Shevchenko.The dynamic of the game was evidently changing and the uselessness of the draw to Kyiv had made the change much more apparent compared to a midseason,run of the mill league fixture.Stoke were becoming more attacking and Kyiv were withdrawing into a defensive shell. When Stoke equalised seven minutes from time they already had five attack minded players on the pitch and were preparing to add more.

With Jones' equalising goal the game's micro climate shifted for a third time,Crouch's intended arrival was aborted and instead a midfielder replaced Jones,Stoke's goalscorer.Walters an attacker with a willingness to help out in midfield was also introduced to help stem Dynamo's all out assault.Had the game been played under an English referee,Stoke would have had to endure at least 5 minutes of injury time,but German officials reputation for adding little or no injury time is well earned and Begovic's world class save in the second minute of added time was the game's last action.

If teams adjust their strategy and thus their likelihood of scoring or conceding goals in matches where game situations are stark such as the Stoke/Kyiv game,the question now becomes does the same thing happen to a lesser extent in more normal games.Are teams becoming more defensive when they lead,if so by how much and is the trade off they presumably receive in fewer goals conceded worth the tactical alterations.We should be able to find evidence for it within the match data and this knowledge will also be necessary if we are to more accurately trace the likely course of a match.

I'll present that data in a future post.

Thursday 1 December 2011

Tottenham 1 PAOK Salonika 2 or Does Football Really Want/Need Six Officials?

A game worthy of a place in Greek mythology ,Salonika's wiki entry already proclaims "The club gained worldwide fame after they defeated European and English superpower Tottenham Hotspur at White Hart Lane".Well maybe that's slightly over stating the case,but there's no doubt that PAOK performed heroically on Wednesday night and overcame the odds against a less than full strength Spurs side.The match was more chaos theory than quality,but it was littered with a couple of major game changing incidents and so let's look at the Expected Points graph to try to make sense of what did and didn't happen.

0-2,Red Card and Penalty awarded,Stafylidis (PAOK),37'
1-2,Penalty Scored,Modric,38'

Spurs made nine changes from the weekend's EPL action and played like strangers for the first quarter of an hour.PAOK's 2 goal lead was fully deserved and for once Gomes was largely blameless.However,Spurs had enough quality and experience on the pitch and on the bench to be far from a forlorn hope.PAOK were certainly favourites,but not massively so.

The first controversial decision came in the 37th minute.Stafulidis blocked the ball on the line with his chest initially and finally with his arm.The ball went from chest to arm almost instantaneously and there was no intent on the part of the defender.The award of a penalty and a red card looked harsh,but the decision was rendered moot before the ref had blown his whistle when Defoe was clattered from behind as he tried to turn the loose ball into the net.Overall the correct ultimate decision,for the wrong incident resulting in the wrong player being dismissed.The award and red card alone were enough to propel Spurs back to favouritism to take more points on average from the game and Modric pushed their supremacy higher still by rolling the spot kick to the keeper's left a split second after he had committed to diving to his right.

Inevitably,Spurs then enjoyed lots of possession and opportunity to equalise,but a combination of packing defence and more desperate and speculative shooting meant that 10 man PAOK stayed in front on this occasion.It also showed how teams tend to see their shooting efficiency decline when they are behind.(See here).By the time the second major incident occurred,a combination of PAOK's resolute defending,Spurs' wayward finishing and the clock had allowed the Greeks to reclaim favouritism.

The timeline of the incident is fairly clear.In the 78th minute,Conteras cleared Falque's shot off the line,he was then fouled by Defoe as he completed the clearance and was prone,injured and playing Defoe onside just in front of the goalline when sometime later the ball was returned into the box and Defoe "scored".Initially the referee seemed happy to allow the goal,but after justified complaints from PAOK and the extremely belated intervention of the extra official behind the goal he awarded a free kick to the Greek side.The decision was almost certainly correct,too many forwards have escaped punishment for late lunges on defenders that would likely result in a card if the roles were reversed,so it's nice to see parity returning to that part of the game.But the mechanism to get to the decision was wretched.

The more people charged with making a judgement call on the same incident,the more opportunity for such a chaotic mess to unfold.If the referee sees an incident and doesn't deem it a foul,should he really be expected to cede to a 5th official who saw the same incident and thought it was a foul? A referee and his assistant differing in their interpretation of incidents is really the maximum amount of potential for farce the players and the paying customers should be expected to put up with.Some sports need a host of officials.The NFL for example needs one official to watch every eligible receiver on every play,football/soccer should stick with three.One sole arbiter and two assistants.On this occasion the correct decision was probably made,but at the cost of turning the match into a protracted pantomime where the audience had little chance of comprehending what had just taken place.

For the record if Defoe's goal had stood Spurs would have averaged just under 2 points from that drawing game position compared to PAOK's half a point.A night to remember,but not for outstanding football.