Mike Dean and Arsenal

Mike Dean takes charge of Saturday's match between Stoke and Arsenal at the Emirates. Dean is no stranger to officiating Arsenal matches and his appearance should give rise to a sense of foreboding for every fan of the Gunners, if a recently circulated statistic on Twitter (@StefanKelly and Rt'd by @RWhittall link) has any credence. In the last 16 Arsenal matches officiated by Dean, Arsenal has won just twice. (The stat actually claims one win in 15, but omits the late season win over Wigan that finally, after years of trying, saw Wigan relegated). Like referees, we all make mistakes.

At first glance the figures appear compelling, over the same timescale, the rest of the top teams from Spurs upwards have each won at lest half of their Dean controlled fixtures. But as is often the case with good Stoke City omens, the case for an assured victory, possibly courtesy of any anticipated generosity of spirit from Mr Dean, is an incredible weak and contrived one.

The initial use of win percentage instead of the more convincing success rate, which includes and weights draws as being worth half a win, artificially extends the apparent gap between Arsenal's fate under Dean and that of their usual rivals. 7% of wins (actually 13% when the Wigan game is included) appears a very long way away from the 50% win figure of Spurs, the next most unfortunate top club to apparently suffer at the hands of Dean.

Mike Dean and Arsenal. 2009-2013.

Sides Reffed by Mr Dean.	Wins.	Draws.	Losses.	Win%	Success Rate.
Arsenal.	2	5	9	13%	28%
Man Utd.	8	2	2	67%	75%
Man City.	10	4	2	63%	75%
Chelsea.	10	4	2	63%	75%
Spurs.	8	4	4	50%	63%

If we include draws, which Arsenal accumulated at the highest rate of the five teams, the resulting success rate slightly narrows the performance gap.

Secondly, the 16 Arsenal matches are a highly correlated set of fixtures. 11 of the matches are against the other top four sides listed in the table. United finished above Arsenal in all four seasons listed, while United's near neighbours and Chelsea also gained more points than the Gunners in three of the four years. Only Spurs proved to be consistently inferior to Wenger's side. Therefore, if Arsenal's rivals perform well in this table, Arsenal, as the team the other sides frequently meet, are virtually guaranteed to do badly. In short, under the selective rules of these engagements, the Gunners are playing tough, superior sides and only wins count.

However, the biggest give away in this attempt to suggest a correlation between Mike Dean and Arsenal's poor results is the selective cut off point made at the start of the 2009 season. If we go back further and include two more games, Arsenal avoid defeat against both Spurs and Manchester United. Include a further game, away at Chelsea in November 2008 and Arsenal record a win, despite Dean's presence in middle. Add one more and Blackburn are comfortably defeated, then Manchester United are held goalless.

If you include all of Dean's missing assignments involving Arsenal, 21 extra Premiership matches show up and Arsenal, apparently unaware that they are facing their future arch nemesis, lose just once and Dean appears to be more of a lucky mascot.

To finally nail this absurd fallacy that referees are able and motivated to determine the outcome of matches involving certain sides, here's a plot of the distribution of the likely success rates achieved by the Arsenal/Dean combination over Dean's Premiership career, once the quality of opponent is allowed for.

Arsenal would have most likely expected to return a 62% success rate over those matches and they actually returned a below average sr of 57%. The chances of the pre game match up ratings being broadly correct and Arsenal recording a record of 57% or worse is a shade under 20%. So there is nothing remotely significant to see here.

Arsenal's below par return is down to random variation rather than a Machiavellian plot. But even if a strong, unlikely to appear by chance, correlation between Dean and Arsenal under-performance is proven, causation then has to be established as well. And other than anecdotal opinion, usually based around red cards, fishing trips of this type rarely attempt to substantiate the headline with such evidence. Can Dean really be held responsible for Koscielny's decision to rugby tackle Dzeko five yards from goal, rather than choosing to defend the ball in an ultimate, 10 man Arsenal defeat to Manchester City?

Even if we allow the selective start points to stand by assuming that Arsenal did something to really annoy Dean at the close of the 2008/09 season, the conspiracy still fails. Arsenal's 28%, opponent adjusted success rate has around a 1% chance of occurring through short term variation, but Arsenal weren't the only side playing under the gaze of a familiar ref in those four seasons.

The Premiership referring pool is relatively small, most of them can squeeze around a small breakfast table at the FA's St George's complex. So lots of sides regularly see different officials over a fairly regular timeframe. Sooner or later, even with odds of 1% for particular combinations of ref/club, one such pairing is going to throw up a seemingly unlikely run of good or bad results, just by chance.

The good news for Arsenal is that Mike Dean hasn't held a grudge against them since 2009. The bad news for Stoke is that if they win on Saturday, it will come unassisted and on the back of their new found passing game, rather than as a gift from the officials. And I'm sure Mr Dean, should the wider press pick up on this absurd "factoid", is professional enough not to attempt to redress an imaginary imbalance in his dealings with Wenger's side.

Where Have All The Strikers Gone.

Having written around 318 posts loosely based around the effects of random variation on sports data in general and football stats in particular, yesterday's post, which tentatively suggested that the lack of early season goals may have various causes in addition to the ubiquitous fluctuations of real time events, attracted the second most daily views in the life of this blog.

C'est la vie.

So as a rapid follow up, I will go through both the thought processes and methodology that went into posting yesterday's post.

Around November 2011, it suddenly dawned on everyone that the Premiership was seeing lots of goals. The big clubs were handing out the type of beatings to their own kind that they usually reserved for lesser teams. Manchester United put 8 past Arsenal, were then tonked 6-1 by City, who had already put 5 past Spurs, before Arsenal replied to Chesea's three with five of their own. And so it went on.

By the time 100 matches had been played, the average number of goals per game was very nearly three. To satisfy myself that we were seeing mostly random variation I simulated the 2011/12 season's first 100 matches to see how often you should expect to see 295 goals if the goal expectancy for each individual match was similar to recent historical levels for the 20 EPL sides.

Unfortunately, Liverpool's Ian Graham beat me to it and posted this when he was still at Decision Tech. Ian's a nice guy, so I'm sure he won't mind me using his post as an example of what I also did back in 2011 and latterly in 2013.

Having been beaten to the post in 2011, I repeated the process using the first 40 matches of the 2013/14 season, fully expecting the results to show that 74 (now 78 goals) in 39 (now 40) matches was likely to arrive about 20% of the time in a simulation using normal, for the current time, goal expectancy. There was a similar 17% chance that a "normal" group of EPL sides had produced 295 goals up to November 2011 purely through random variation and I fully expected to finally get to post my 2011 effort, two years late.

Instead I got a figure of 0.5%.

You are of course free to set the bar of "proof" as high or as low as you wish, but the 17% chance of 295 goals appearing in 100 EPL matches if we assume scoring is around historical levels, strongly implies short term fluctuation is the most likely, major cause. There is no need to speculate on why the art of defending is in decline or striking is in the ascendancy. The 0.5% figure for the first 40 matches of 2013/14 to produce 78 goals due to short term fluctuations of a typically higher recent goal expectation, however carries a much larger requirement for blind faith.

An alternative viewpoint may be that for various reasons, the current EPL over the course of the first 40 games has been played out by teams that may only be capable (or intend) to score less goals. If you simulate a 40 game run of matches where the average goal expectancy for each match is just below 2.5 goals, the chances of a total of 78 goals appearing jumps from 0.5% (using the historically higher goal expectancy) to around 10%. This figure is a much easier "sell"

The last four EPL seasons have averaged around 2.8 goals per game, prior to that and further back into the 90's, figures in the region of 2.5 over a whole season aren't uncommon. The French premier league regularly throws up total goal averages over a season that are even lower at around 2.3 goals per game. The opening group matches for the last five European Championships (conveniently totaling 40 games) has produced 2.25 goals per game compared to 2.78 goals per game for the final group matches. So lower average goal totals aren't unusual in top flight football and France.

So we may be seeing random variation (we always will), but around a lower expected goal scoring average. And that is where observation becomes as important as statistical analysis based around assumptions of what has passed for recent normality in the EPL.

It has been suggested that nothing short of a mass suicide of strikers could account for the expected goals total falling to 1.95 a game. But it need not fall to that level, it just has to fall compared to previous highs and then experience short term variation to arrive at a figure that sits to the left of the true average, but happens with enough frequency (say around 10%) to provide a believable explanation for real events.

Minutes Played out of a Possible Maximum of 3240 By the Major Scorers for Eight EPL Sides in 2013/14.

Team.	Player(s).	Goals scored in 2012/13.	Goals as a Percentage of Team Total.	Minutes Played in 2013/14.
Tottenham	Bale/Defoe	32	51%	37
Crystal Palace.	Murray.	30	41%	0
Liverpool.	Suarez.	23	34%	0
WBA (Loan).	Lukaku.	17	34%	40
Newcastle.	Ba.	13	30%	65
Norwich.	Holt.	8	20%	0
Manchester City.	Tevez.	11	17%	0
West Ham.	Carroll.	7	16%	0

All EPL goal scoring strikers are currently alive and well, but many are in new (non) Premiership surroundings or idle.

Bale is of course in Madrid and Defoe, with whom he combined to score over half of Spurs' goals in 2012/13 is an occasional late substitute. Palace's 30 goal striker, Glen Murray is still recuperating from an injury sustained in the playoff semi final. Holt is now in the Championship, Suarez is completing an anger management course, Lukaku is largely practicing penalties on Chelsea's training ground prior to his loan to Everton and Ba is similarly underused, Carroll is injured and Tevez has departed.

So we can chose between random, short term variation around a historically high goal expectation leading to a very rare run of scorelines or (and not necessarily the only alternative), short term random variation around a slightly lower goal expectation caused by the temporary or permanent unavailability of 141 goals worth of talent, but likely to occur much more frequently.

The general opinion is that levels at the end of the year will return to more normal totals, but that won't validate the opinion that everything was "normal" in August and September. The conditions of the trials (games) will inevitably be different also. Suarez can return in a week, Lukaku will be unleashed on defences by Everton as he was by WBA in 2012/13, Carroll and Murray will return from injury and Ba, another talented scorer with little appreciation at Chelsea, may even get to wear in anger that Stoke shirt he wore for his (failed) 2011 medical.

I'm a huge advocate of accounting for short term, random variation, but it shouldn't be used exclusively to solve every conundrum. 

  

A Premiership Goal Drought.

Stoke fans will take their team's slight elevation in the Match of the Day running order as a sign that Lineker and Co. are finally acknowledging that the Potters can, if needed play relatively attractive football. An alternative, and possibly more persuasive argument would be that the Premiership as a whole has produced so few goals in the first 39 matches, that even a relatively mediocre game can jump up the pecking order.

Seasonal scoring trends has been a feature of both the EPL and football league in the past. Goal scoring traditionally started slowly in August, quickly rose to a peak, before falling gradually over the winter months, before a May goals fest. However, in scoring just 74 goals in 39 matches so far, the EPL has set a low scoring precedent.

The average scoring rate in the EPL hovers around 2.5 goals a game, it had averaged 2.64 since 2002 prior to the start of this season. But bouts of lower scoring are also inevitable, especially over relatively small sequences of matches. If you split the EPL since 2002 into weekly blocks of 40 matches, around 1.5% of those 40 match runs produced an average of 2 or fewer goals per game. So on this basis the current run would appear to be certainly rare, but not entirely precluded.

Teams can, fairly predictably, produce low scoring matches over relatively short time frames. The opening group matches of a major tournament, such as the World Cup for example or hugely important one off matches such as FA cup finals, tend to be less goal laden than earlier rounds, in the case of cup ties and later games, in the case of group matches.

However, generally the gap in quality between each side is the major factor in determining how many goals will be scored in a match, especially over league scale time frames. The remaining week four match sees Liverpool travel to Swansea as narrow, 4 tenths of a goal favourites, so you would expect around 2.6 goals to be scored in this fixture, on average. A more of a miss match, along the lines of Palace's visit to Old Trafford would see the total goals line creep just past 3.

It is therefore possible to estimated, based on historical general scoring rates, as well a individual team tendencies, the average number of goals each of the previous 39 games would expect to see and from this information, the range and frequency can be simulated for total goals scored in those 39 games. The most convenient way is to model the sores via a Poisson distribution.

Although it is tempting to lump the first 39 or 40 matches of each season together as a single repeatable trial, there are small differences caused by the slightly different team strengths and match ups thrown up by the fixture list compilers, from one year to the next. The 2013/14 season has seen slightly more closely matched games, based on pre game estimates, than has been the case recently. Therefore, we should expect to see these more frequent, potentially closely fought matches produce less goals per game. However, this effect is unlikely to wholly account for the figures that we have seen.

The relative rarity of so few goals being scored in actual batches of 40 matches is repeated in the simulation for the 2013/14 season to date. The simulations using goal expectancy figures for all sides that would be consistent with figures seen over the recent EPL history, generated a season with 74 goals in the first 39 games once every 550 seasons. While a season that had 74 or fewer goals in the first 39 matches appeared once every 185 campaigns.

So, although not precluded from occurring under the kind of Premiership football we have witnessed over recent seasons, the first 39 matches in 2013/14 are extreme outliers.

We've touched briefly on matches where sides capable of producing games with "normal" goal totals can tactically adapt, possibly through fear of losing, to produce much lower scoring games, such as the opening group matches of a tournament. So perhaps other factors, such as these are present in the 2013/14 Premiership, other than the natural random variation within small sample sizes.

Five clubs do have new managers and one, Mourinho at Chelsea, consistently presided over low scoring matches in his previous stint at the club, under a policy that coveted defensive strength. And as in most sports, defence is more readily organised and excelled at than is creative attacking play. Numerous other managers are also relative newcomers or are now managing in an elevated arena compared to the Championship.

Secondly, almost half of last season's top twenty Premiership scorers have barely kicked a ball in anger, so far. Some, such as Bale and Tevez are no longer Premiership players, while prolific scorers, such as Ba, Lukaku, Suarez and Defoe are yet to start a game. Also, depending on source, shots are currently hitting the target with nearly 40% less regularity, compared to historical levels.

Overall, every team, with the exception of the surprisingly good, Arsenal and the predictably chaotic, Sunderland have been involved in matches that have seen their total match goals come in below the expectation predicted by an, until now, robust Poisson based modelling approach.

It is easy to weave narrative when presented with unusual outcomes. The goal glut at the start of 2011/12 didn't continue, although the chances of it appearing, with little change in scoring intent, was a much more substantial 18% compared to the present paltry 0.5% for our current drought. For once sample size and random variation doesn't appear to Bale out those looking for a mundane explanation, although it is certainly a component of the sought after solution.

Time to test the theories!