Where To Draw the Line for Premiership Survival.

The expectations of football fans at the start of the season can be broadly split into two camps. Supporters of around half a dozen teams can look forward to a season where the anxiety revolves around title aspirations and UCL or Europa league qualification. Some even organise a flypast to celebrate the latter. The remainder of have-nots can't wait to get to the magical 40 points quick enough and reach the generally accepted threshold for continued membership of the top flight.

The 40 point target usually leaves quite a bit of room for error, the mid to high thirties has generally been sufficient to finish outside the bottom three throughout the history of the 38 game Premiership. Only West Ham (42 points) in 2002/03 have been relegated with more than 40 points, although Everton needed a superior goal difference to go with their two score points to survive in 1997/98 and Sunderland found 40 points insufficient a year earlier.

Each of these seasons were atypical of the current EPL. Everton's near miss-hap and Sunderland's demise came in a period when fewer than 80 points were needed to win the title, so "extra" points filtered down the league and West Ham suffered partly through the generous nature of the other two relegated sides, WBA (26 points) and Sunderland (19 points).

This demonstrates that while historical precedence can give a baseline figure, factors unique to particular seasons also wield some influence.

If the likes of Derby, Coventry and Sheffield Wednesday hadn't embarked on unlikely goal scoring sprees against Bolton in 1997/98, Everton would have suffered relegation on goal difference instead of clinging to their top flight status.

And if WHU had swapped an impressive 3-2 win away at Chelsea for a much more valuable win at lowly Bolton in 2002/03 instead of the narrow 1-0 defeat they actually suffered, the Hammers 42 points would have been sufficient and Bolton would have dropped into the Championship with 41 points.

Therefore, goal difference, an intertwined fixture list (14 of the remaining matches are head to heads between teams currently occupying one of the bottom nine places) and the proximity of the relegation candidates to each other, each add layers of uncertainty to the survival target of the threatened sides. And these effects become more apparent as the season winds to an end.

The case of Crystal Palace illustrates some of these factors. Their run in is tough and two matches away at both Cardiff and Fulham present them with one of their best opportunities to pick up points in their remaining matches. If they do take points from these games they will deny one or more of their pursuers the chance to take maximum points. Additionally they have a superior goal difference and this is likely to be the case after 38 games.

Palace's goal difference under Pulis' 20 matches in charge is -4, compared to -15 in the first 11 matches before his arrival. Even if they only limp to 34 points by May, simulations indicate that they will still have just over a 50% chance of beating the drop.

34 points is not only at the lower end of the points estimate for Palace by the end of the season, (their most likely total is 38 points), but it also gives them a better than even money shot at staying up.

But what about their relegation rivals, Sunderland, currently lying in 19th place with 25 points from 30 games. Palace are an obvious target for Sunderland, but their fixture lists don't cross again this season, although their goal differences are very similar. However, they do play WBA whom are also struggling at the foot of the table, but the Baggies have an easier run in than Palace, as well as a superior goal difference. To complicate matters further, Cardiff also visit the Stadium of Light at the end of April.

If Sunderland reach 34 points, (an over-achievement on current evidence), how will all these intertwined factors play out in terms of their survival chances?

The easiest way to unravel this is to again simulate actual seasons and see how often 34 points sees Sunderland safe compared to others marooned at the foot of the table.

The table above appears to indicate that rather than a single survival line at this late stage of the season, each team has its own individual target. The chances of enough teams emerging from the bottom of the pack to relegate Palace if they end the campaign with just 34 points is around 53%.

In contrast, the likelihood that Sunderland will find enough sides to catch should they gain 34 points in a multitude of different iterations of the remaining 64 matches is only 30%. Cardiff, especially with their poor goal difference, need a near superhuman 36 points before they are more likely than not to remain in the top flight, as most of the birds will have flown to a slightly higher perch.

The playing field is level in August, but by May the really struggling teams appear to be required to clear a slightly higher bar just to give themselves a similar shot at surviving the drop.

*although Fulham aren't on the (slightly cluttered) plot, they are of course included in the simulations. Their potential fate is slightly worse than that projected for Cardiff.

Judging A Side Over One Season.

For much of Alan Pardew's reign at Newcastle, his side has provided ample topics of discussion for the analytical community. Their fifth placed finish in 2011/12, with a goal difference of just 5, was widely seen as the product of a decent side getting a bit lucky and their near relegation the following season merely served to reinforce some misconceptions that exist around luck based over-achievement immediately snapping back into ledger balancing under-achievement.

This season the team has largely occupied the dead ground between Simon Gleave's Superior Seven and Threatened Thirteen. They currently hold 8th spot with very little likely up or downside, casting them as mere spectators to the title and Champions League tussles and the relegation grind. So it has been left to their off field employees to guarantee column inches.

Over the previous two and three quarter seasons, Newcastle's success rate, that counts draws as half wins, has averaged out at a shade over 50%. If the Premiership were allowed to meander on without a season's end reckoning, Newcastle since the start of 2011/12 have recorded results typical of a side finishing a 38 game season in 8th or 9th spot, with a goal difference in the region of zero. So, despite the bouts of squad strengthening and weakening during the various windows, might Newcastle have been broadly a "best of the rest" team under Pardew and has their seemingly up and down seasons simply been down to talent and luck interacting in differing doses?

Simulating season long results for one side or a whole league can quickly produce the range of possible outcomes we might expect from teams of a certain quality and it is perhaps a sobering thought, especially for managers that there is a non trivial chance that the best side from the Threatened Thirteen could find itself struggling to top 40 points over a 38 game campaign.

There is almost a 10% chance that a side with a true success rate of 0.5 may recorded 41 or fewer points, the limit of Newcastle's achievement in 2012/13 and a 6% chance that they could rise to the highs achieved in the previous season.

So although a dissected break down of Pardew's previous two completed seasons would give an entirely factual record of the points per game performance of his side in the 2011/12 and 2012/13 seasons, we can be much less sure if that see sawing record accurately tracked the talent of the team.

What we see, isn't a cast iron endorsement of quality or the lack of it, but a combination of randomness and talent.

It would have been easy to over react to the raw results posted by Newcastle in either season and while injury, Europa League involvement, squad churn and tactical change almost certainly impacted on their talent and luck based achievements, it seems unlikely that the 24 point difference between the 2012/13 team and the 2011/12 team truly expressed the quality gap between those two closely related editions of the side.

One way of trying to reconcile an apparent points based performance improvement to any real shift in a sides actual abilities is to utilize to expertise of the bookmaking industry. At the start of the season all of the spread firms publish the expected number of league points they think each side will gather by the end of the season. As the season progresses these estimates will be up or downgraded in line with each side's current rate of accumulation of points.

For example, prior to last Saturday's eventful head to head meeting with Newcastle, Hull had already posted 31 points, almost equal to the general preseason estimate of around 33 or 34 points. Their updated quote suggested they might get about 42 points. So from a performance perspective they were roughly 8 points ahead of where most bookmakers expected them to be.

We have also reached the second half of the season, where a side increasingly completes their head to head appointments and by mid March a team has played a rival home and away between 9 and 11 times. The chances of a team winning or drawing a game depends on not only the gap in quality between the teams, but also the venue. So the expected success rate in a return fixture will depend partly on how each team has progressed or regressed over the 190 or so days between the initial meeting, but also where each match was played.

Above, I've plotted the pre-game quoted success rates for actual Premiership matches, paired by home and away matches. So a home side that is expected to have a 60% success rate at home would likely be quoted at around 40% when they traveled to complete the reverse fixture.

If we look at the actual differences in the quoted success rates for return matches any deviation from this fairly strong, general relationship can be, at least partly attributed to the direction both sides have moved in over the ensuing timescale.

The case of Crystal Palace (or more accurately, Tony Pulis) demonstrate the process. Up until March 19th, they have played 10 teams twice. Stoke, Hull, Manchester United, Norwich, WBA and Southampton each made a return visit to Palace and Palace set off to reacquaint themselves at the homes of Arsenal, Spurs. Sunderland and Swansea.

In eight of those ten matches the quoted pregame success rate in the return was greater than predicted by the success rate in the initial game and the line of best fit from the plot above. Palace's quoted success rate posted by the bookmakers is on average 15% higher over the ten rematches, compared to a typical example from the line of best fit. In short, the recent quotes imply a level of improvement compared to Palace in the opening months of the season.

There are strength of schedule issues, Palace may have played ten teams whose true ability has dipped, but this can be addressed by a least squares approach to the overall loss or gain in success rate of each side, when measured against their actual repeat schedule. The order shuffles slightly if you apply this correction.

Which Teams Have Impressed Enough to be Upgraded from Preseason Estimates?

Team.	Average Shift in Success Rate per Game.
Crystal Palace.	0.06
Liverpool.	0.04
Hull.	0.04
Arsenal.	0.04
Manchester City.	0.04
Chelsea.	0.03
Everton.	0.02
Newcastle.	0.01
Sunderland.	0
Southampton.	0
Manchester U.	-0.01
Spurs.	-0.01
WBA.	-0.02
Norwich.	-0.02
Stoke.	-0.04
Swansea.	-0.04
Cardiff.	-0.04
Aston Villa.	-0.04
Fulham.	-0.05
WHU.	-0.06

The table above shows the strength of schedule corrected, average raw success rate increases or decreases per game as seen in the repeat fixtures played so far by each Premiership team. It's a combination of perceived improvement (Liverpool) or decline (Manchester United) and partly a popularity contest.

Stoke are on course to better most preseason points estimates, but their marmite factor makes them a universally unpopular team, which may account for their apparent downgrading, despite a better than expected transitional campaign. Even punters have to be offered inducements to side with Stoke! But hopefully, most sides in the table have been up or downgraded predominately on the bookmakers interpretation of repeatable form.

To add context to the previous table, I've finally plotted the change in pregame quoted success rate against the rise or fall in expected final league points totals compared to the estimates at the start of the season. Fulham, for instance are on course to be 10 points adrift of preseason estimates and they've been downgraded by an average success rate of ~0.05. Slightly more than you would expect from the plot.

Palace are on course to gain about 36 points compared to the 31 quoted in August, but their position, well above the average line appears to reflect extremely well on current manager, Tony Pulis and less so on Ian Holloway, who was in charge for the bulk of 2013. Pulis, once again demonstrates an ability to turn relegation fodder into lower half scrappers who can take their season to the wire. When he was appointed Palace had 4 points from a possible 33 and were rock bottom.

Steve Bruce's tenure at Hull is also endorsed by the betting movements. Not only were the Tigers underrated, but there appears cause for added optimism above and beyond there current position of relative safety.

Both Merseyside teams have gather more points than expected, but Everton's upgrade still lies below the line of best fit, possibly indicating a reluctance to take their improvement wholly at face value. A caveat that also applies to Southampton.

Liverpool may be the most telling point on the plot. They were expected to gather around 66 league points when the season began, currently they may be able to top 80 points. An increase of around 16 points. However, if we roughly apply their average increased success rate that is currently being applied to them over a 38 game season, their points total increases by just around half of those 16 points. Outstanding, but partly unexpected performance isn't being taken at face value on a match by match basis. It is also being taken with a slight pinch of salt.

The Reds' uptick is as spectacular as the 38 game epic demonstrated by Newcastle, when they nearly claimed a Champions' League spot, but it may be well to remember that single season performances are a product of repeatable skills and also less repeatable variation. Even with points in the bag, a team might not really be quite as good (or bad) as their record might indicate.  

Crystal Palace. Just Like Watching Stoke City.

The sighting of a Pulis led side that was out shot, out possessed and out passed, yet still managed to take all three points was hardly a new experience for the 2,000+ Stoke fans making the trip to Selhurst Park on Saturday. This tactical wrinkle, based firmly around allowing poor sides the ball until they run out of limited ideas, before succumbing to either a set piece score or a mental error worked as well against Pulis' former side as it had against their exasperated opponents during his memorable time spent in charge at the Britannia Stadium.

Post match reaction of fans from the Potteries, following a largely dour and unattractive 1-0 defeat, ranged from the ironic "How can they watch that every week ?" to "I'd take him back tomorrow". All told, not a great away day, as the bottom of the table contracted even further.

It hasn't taken Pulis long to install his preferred approach and the most obvious change can be seen in the frequency at which the players are now making passing attempts compared to their rate of passing first under Ian Holloway and also under interim coach, Keith Millen.

Average Time Elapsed Between Passes Under Holloway/Millen and Under Pulis.

Player.	Mins/Pass Before Pulis.	Mins/Pass With Pulis.
Dean Moxey	3.0	4.8
Mile Jedinak	1.8	2.4
Damien Delaney	3.3	4.2
Danny Gabbidon	3.3	5.6
Joel Ward	2.4	3.1
Marouane Chamakh	2.4	2.7
Kagisho Dikgacoi	2.1	2.9
Dwight Gayle	2.0	3.7
Jason Puncheon	1.9	3.2
Adrian Mariappa	3.9	3.7
Barry Bannan	2.3	2.9
Cameron Jerome	3.8	4.1
Yannick Bolasie	4.0	4.5

Virtually every Palace player whom has played substantial time in 2013/14 under Ian Holloway/Keith Millen and now under Pulis has seen their playing time adjusted passing rate subsequently contract. It is always possible that random variation across sample sizes, combined with different opponent strength can produce similar effects. However, so extreme has the change been across all players, coupled with Pulis' previous preferences, there can be little doubt that there has been a major shift in emphasis. The chances of both sets of passing statistics being drawn from a common ancestor is remote, the differences are significant.

Palace players are making fewer passes per minute under Tony.

Saturday's match also produced a study of the type and frequency of shots at goal typically demanded by Mark Hughes at Stoke and Tony Pulis at Palace. Despite being comprehensively out shot by 17 to 12, Palace still amassed a slightly better cumulative goals expectation when x,y shot location was accounted for. 1.2 expected goals for the hosts compared to 1.1 expected goals for Stoke.

The culprit and recurring theme for Hughes coached teams was an abundance of long range efforts from the visitors. 12 of Stoke's 17 efforts from distance individually had a (much) less than 8% chance of producing a goal compared to just  just 6 for Palace. The hosts also dominated the high return efforts closer to goal, another Pulis trait carried over from his time at Stoke. Three Palace efforts had an individual goal expectancy of 20% or greater, a chance taking area where Stoke drew a matchday blank.

































Three of Palace's high value opportunities came by way of a triple Jack Butland save, an indication of Pulis' fine eye for a keeper, that often deflects attention from his more scatter gun approach to recruiting successful out field talent during his tenure at Stoke. Therefore, if we wish to evaluate the fairness of the result in light of the shots attempted and conceded by each side, we need to acknowledge that, even in is partly artificial probabilistic reconstruction of Saturday's classic, Palace could only have scored once from this intimately related barrage of rebounds and shots.

Despite the larger footballs in use at Stoke, Assaidi is providing an unsustainable number of goals from distance.

Once the shot locations are evaluated, related events identified and outcomes simulated, the actual result falls well within the bounds of likely outcomes. Palace's slightly superior overall goal expectation is eroded enough by Butland's triple save to make Stoke narrowly the more likely to win a prolonged simulation of the soring chances. A 35% win probability based on shots taken for the visitors barely edges the 33% for the hosts.

A 1-1 draw was the most common scoreline seen in the simulations, closely followed by the actual 1-0 win recorded on the day by Palace. Stoke then pile in with a couple of higher overall total goal victories to edge Palace overall, but despite 29 total shots, the match was very unlikely to turn into the eight goal thriller that Stoke had fought out with Liverpool during round 21.

Pulis' relegation avoidance strategy, honed throughout his managerial career, particularly at Stoke is likely to give Palace a decent shot a staying up. Especially against limited opponents, such as the current, devoid of Fuller-esque pace and guile, Stoke team, against whom his strategy appears to work best.

Pulisball. What to Expect at Palace.

Crystal Palace's long search for a new manager finally ended on Friday night, when one of the early front runners, Tony Pulis put pen to paper. Any lingering doubts about Pulis' at times ugly style of play was ultimately trumped by a managerial record that has never seen him relegated. Sacked yes, but not relegated.

Pulisball gained ridicule, contempt, grudging respect (and that was just from his own fans), but just enough points to guarantee perennial survival. With the unique ingredient provided by Delap's long throw it did prove an over achieving way of retaining Premiership status for a largely Championship quality team.

Long throws, long balls and set piece goals were the most visible face of a Pulis led Stoke, but it wasn't the only requirement if 40 points were to be achieved year on year. It also needed a defence. A side that relies on set pieces to provide a higher that average proportion of their goals are usually limited attacking sides. The correlation isn't overwhelming, but the trend exists for sides that score proportionally more goals from set plays, to also score relatively fewer goals over a season compared to sides that have a more balanced attacking approach.

Unsurprisingly, an inability to score goals also goes hand in hand with lower end of season points totals and the strength of the correlation is very strong. The r^2 values for success rate (a proxy for points) and goals scored since the start of the 2008 season is 0.77, so 77% of the variance in success rate comes about as a product of the variance in goals scored.

A slightly weaker correlation, but of a similar order of magnitude for r^2 also exists between goals allowed and seasonal success, which leads to the inevitable conclusion that goals scored and goals conceded by sides are themselves not independent. The negative co dependency between the number of goals a side scores and the number they concede is plotted below. The more goals a side scores in a season, the fewer they will tend to concede.

This correlation is probably driven, firstly by a general difference in relative ability between sides. Better sides are simply more adept at playing all aspects of football. Such sides score more and concede less, but there may also be a tactical aspect. It is difficult to score yourself when your opponents has the ball in your final third. In some cases, attack may also be an effective form of defence.

So the way a side either chooses or it forced to try to accumulate goals in the EPL can have an effect on the amount of goals you score. Stoke's reliance on set play goals was likely to result in few goals being scored. The poor goal totals would in turn lead to unimpressive seasonal success rates and that in turn often led to relegation at worst and bottom half finishes as the norm.

However, Stoke ended four of the five seasons under Pulis with higher seasonal success rates compared to the line of best fit for goals scored, denoted by the red points in the second plot. A clue to why this might have occurred can be seen in the plot above. In all years under Pulis, they allowed fewer goals than expected for a side that scored at the rate they did. Again Stoke's five seasons under Pulis are represented by the five red points in the plot above. The brutal simplicity of creating a set piece chance just by winning a throw inside the opponents half, often meant that Stoke didn't have to commit many players forward to do so. The tactical aspect of Stoke's chance creation left plenty of defensive resources intact to keep the score down in games that mattered.

We can't dismiss the possibility that Stoke's relative defensive excellence was just merely down to chance. If we look at enough sides, some are going to appear continually better than expected, but by pure chance. However, anyone who has watched Stoke, especially in the early EPL seasons can't have failed to notice the overtly defensive stance they took both at home and away from the Potteries.

Although they were correctly classified as a side that relied on set pieces for the majority of their goals, it would be a mistake to look at this aspect of their play in isolation. A team is a sum of their parts and the general outlook for set play sides is rather bleak. They are more likely to be relegated than a side that can score proportionally more often from open play. They finish in the bottom half of the table with greater frequency and they are highly unlikely to be capable of challenging for even a Europa spot.

Pulis' Stoke ticked most of the statistical boxes for a limited, set piece reliant side. Palace already score nearly half of their goals from that source, although raw numbers are small, so percentages will fluctuate. But the real wrinkle that Pulis introduced that propelled an approach with little upside or room for error to a relatively comfortable way of retaining your top flight existence, was his insistence on a defensive team effort.

Palace are already a side that find scoring difficult, so their limitations already provide half of the ingredients for Pulisball to make a Premiership return. To complete the formula, Pulis' secret sauce will be liberally applied to the defence in an effort to maintain his proud record.... but this time he'll have to do it already 12 games into a season and without a defence enhancing wildcard such as Rory Delap.

Stoke 2 Crystal Palace 1. A Shot Analysis.

Crystal Palace's visit to the Britannia Stadium pitted together two of the favourites for a return to Championship football in time for the 2014/15 season. As one of the promoted sides, Palace's position is understandable, especially with the loss of Zaha to the champions, Manchester United and an elevation that came by way of the playoffs. Their chance of an immediate return is currently rated around the 70% mark.

The experts are less confident in the prospect of Stoke slipping back down the pyramid come May. In contrast with the pessimism surrounding Saturday's recent opponents, the Potters currently enjoy the considered consensus of a 75 to 80% chance of remaining in the top flight. That number still makes them fourth favourites to be one of the three relegated sides. A sign of the uncertainty surrounding a new manager in Mark Hughes, a seismic change in style and the competitiveness of the lower reaches of the Premiership, rather than an acknowledgement of Stoke as an established EPL team.

So the match on Saturday was an early season opportunity for potential strugglers at the lower end of the table to pick up valuable points, while guaranteeing none for their likely rivals. A nervous 90 minutes awaited both sets of fans, especially Stoke's, who as the home side were reasonably strong pregame favourites and had a manager whom, as a high profile pundit explained "hadn't won a Premiership game since January" (total sample size, one match, away at Anfield).

With the week's marque matchup taking place on Monday night at Old Trafford and the remainder of Saturday's Premiership entertainment providing less than riveting viewing, Stoke's 2-1, come from behind victory appeared second on Match of the Day. As is usual in such truncated highlights, most of the action featured the goals and various goal attempts from each side and initial impressions appear to show that Palace might consider themselves unlucky losers on the balance of attempts.

                                                 Stoke Attempts on the Palace Goal.

                                                 Palace Attempts on the Stoke Goal.

Both sides had 14 attempts on goal with each side requiring the other to make an intervention to a goal bound shot on 5 occasions. So was Stoke outscoring Palace by 2 to 1, not only an absurd use of ratios, but also a fortunate outcome for The Potters and would a share of the spoils been a more appropriate final outcome?

The use of shots and headers to determine the fairness of an actual result is rife with flaws and contradictions. A side tailors their attacking approach to the current game state and that usually means more goal attempts when that state is an unfavourable one. So shot numbers are a function of what actually happened and in which order, on the day. Also a different outcome for one attempt would inevitable lead in reality to later chances being stillborn and others being spawned.

Nevertheless, it  may still be a fruitful and illuminating task to assign each actual attempt a likelihood of it resulting in a score, once factors such as pitch position for the shot, if it was with the boot or the head and if, as was the case for Palace's goal, if it came via a counter attack (or aimless punt down field as it was know when the tactic was extensively used by Pulis' Stoke) are accounted for.


Most Likely Scorelines Given the Quality and Quantity of Palace and Stoke City's Goal Attempts.

Shot Generated Scoreline. (Stoke Score Appears First).	% Frequency.
1-0	25.2
0-0	16.2
2-0	15.5
1-1	11.7
2-1	7.3
0-1	6.7
3-0	6.2
3-1	2.7
1-2	2.1
0-2	1.4
2-2	1.3
4-0	1.3
4-1	0.6
3-2	0.5
1-3	0.3
5-0	0.1
2-3	0.1
4-2	0.1
5-1	0.1
0-3	0.1
3-3	0
5-2	0
0-4	0
4-3	0
2-4	0
6-0	0

* the more unlikely outcomes occurred with such infrequency that they fail to register to one decimal place.

Just over half of Stoke's attempts came inside the box compared to just less than half for their visitors. So initially, even with such a rudimentary classification of attempts, the home side grabs a slight advantage. Better still for Stoke three of their eight penalty area attempts came in an area bounded by two perpendicular lines drawn from each goal post and three others were only marginally outside this lethal shooting area. Once shots from inside the area move towards the lateral extremities of the box, the likelihood of a score begins to fall away. Palace didn't manage one shot from the more profitable central positions.

As a consequence, although numerically equal, Stoke's attempts had a much higher likelihood of a goal resulting from their efforts. The Potters' goal expectancy from their 14 attempts eclipsed that of Palace's 14 efforts by over eight tenths of a goal.

This goal expectancy supremacy, based on the quality of the attempts made by each side on the actual matchday begins to validate the actual one goal margin by which Stoke won the game. But we can go further by simulating the actual outcome of all 28 shots in thousands of possible iterations of Saturday's match. The excitement may be too much for committed Stoke ( @MarkTaylor0) and Palace ( @SimonGleave ) fans to bear!

Over ten thousand iterations, Stoke were by far the most likely winner, judged solely on the quality shot count. They "won" 60% of the simulations, drew 29% and only lost 11% of the time. The five most likely match outcomes all saw Stoke avoiding defeat and the game's actual scoreline was the fifth most likely simulation scoreline, occurring just over 7% of the time. City fans will hope the 25% chance of such a shootout finishing 1-0 doesn't herald a return to the dark, unappealing days of Stoke's binary past.

Overall, less than sparking entertainment, but a fair result in hindsight and three points for Stoke towards their customary 40 point target.