Saturday 13 December 2014

How Well Might Total Shot Ratio in Football Travel ?

One of the most familiar of the crossover stats from hockey to football is total shot ratio. It is often used as a proxy for "dominance" of possession and supposes that a side that is constantly out-shot or out shoots their opponent will eventually reap their just rewards.

The majority of the football based research has been focused quite naturally on the English Premier League, most notably by James Grayson, who demonstrates both the repeatability and predictive nature of the stat for the EPL.

Very little of the detailed validation that James has undertaken for the EPL has been done on any other the other world leagues, but this hasn't prevented a combination of a respectable TSR and a relatively low league position earning a non EPL side team an "unlucky" tag.

If we first consider the case of possession, It has largely been discarded as credible measure of team strength. Barcelona's use of the tactic, with varying degrees of success, suffered at the feet of Bayern Munich in the 2013 ECL semi final.

Spain's finest held the majority of the ball, but fell to a 7-0 two legged thrashing. And in more humble surroundings, Stoke mostly under Pulis, graciously allowed themselves to be continually "dominated" on all fronts, even at home, but still won enough games to safely avoid the drop each season.

Stoke Shrug & Say No to TSR or Possession Stats.
However, despite possession's decidedly mixed record of success, it is possible to construct a scatter plot of possession against match outcome that does appear to present a pleasingly angled line of best fit that appears to show more possession positively linked to more success on the field.

It is only when the top three or four teams are removed from the plot that the correlation for the remaining teams becomes largely random.

The constant presence of a handful of sides which dominate in a league, not only in wins, but in possession and shots by simply outclassing the majority of their opponents, therefore, may potentially create a misleading correlation, where only a weak one exists for the majority of teams.

So might the illusion of possession being generally correlated to results be repeated to some degree, in some leagues for TSR.

The Scottish Premiership has largely been a two horse race between Rangers (until their demotion following their liquidation) and Celtic. The league has an unusual format, comprising of 12 sides and the table splits in two once each team has played three matches against each of their rivals, enabling a 38 game season to be played out.

In common with the EPL, a Scottish Premiership side's TSR in the first half of the season appears to show a strong correlation to goal difference in the remainder. Therefore, TSR appears to do a good job of predicting a useful future quality of teams in Scotland's top flight.

The coefficient of correlation is a very healthy 0.71 for the four seasons from 2010/11 onwards. However, the scatterplot is fairly lopsided, there is very nearly daylight between the six points in the top right hand corner, four Celtic seasons and the only two from Rangers, and the rest.

As with possession, where a strong correlation is inferred because of the constant presence of atypical sides which partly outclass the rest, have Celtic and Rangers exaggerated the implied predictive power of TSR for the remainder of the teams, outside of the "Old Firm" by helping to create an impressive r value?

If we crudely remove the six "Old Firm" points from the plot, the scatter plot and the coefficient of correlation between past TSR and future goal difference for the remaining ten per season Scottish sides alters dramatically.

If we remove Rangers and Celtic, r falls to 0.14 as does the predictive power of the relationship. TSR may still tell us something about the remainder of the season for these non elite sides, but without a highly likely combination of a regular, out shot, defeat by either Celtic or Rangers, our conclusions become much less dramatic.

The separation of abilities in the EPL is perhaps less stark than in Scotland's top flight, but a division along the lines of Simon Gleave's "Superior seven and threatened thirteen" has largely existed over the last decade.

Once again, the coefficient of correlation is high if we include all Premiership sides and although the clustering of teams at the middle and bottom isn't as pronounced as in Scotland, there does appear to be an effect.

If we remove every game involving at least one of Simon's superior seven, we get a truncated season in which the threatened thirteen only contest home and away matches against one of the twelve remaining teams from this group.

If we now split the season in two and regress a threatened thirteen side's TSR in the first half of the season to their goal difference in the remainder of the season, the coefficient of correlation again falls. This time from 0.73 to 0.29.

Admittedly, the sample size has also fallen in both cases when the very best sides are removed, but the confidence with which TSR is used indiscriminately across and perhaps within leagues to evaluate certain types of sides perhaps should be questioned.

Just as the past possession record of sides is largely irrelevant to many future outcomes, (even when we can easily produce a scatter plot between possession and outcome to create the illusion of a strong, universal relationship), previous team TSR might also be much less of a factor in the prediction of some types of games.

In conclusion, we seem to get high r values for the relationship between TSR at half season and future goal difference in the remaining games, in leagues where there are a couple of consistently dominant teams, in every aspect of play.

Barca and Real Madrid in Spain, Bayern Munich in Germany, Celtic and a soon to be reunited Rangers in Scotland, the superior seven in the EPL.

The coefficient of correlation for the previous four full seasons, for EPL, Spain, Germany and Scotland are, respectively 0.77, 0.73 0.68 and 0.71.

In France, where dominant sides are less in number and also less dominant, r is 0.51, in Italy it is 0.53.

And most pertinently perhaps, in the bottom tier of the English football league, where runaway winners are rare, r for TSR to future goal difference from the mid season point over the last four seasons is barely 0.4.

So at the very least, the clear assertion that TSR is a good indicator of likely future performance, in every league and within every league at every level, ranging from the English Championship to the MLS and the Egyptian league to the Australian professional league, should also be backed up by the kind of thorough and rigorous validation that James has carried out for the EPL.

Without that, the strength of any possible correlation can only be guessed at.


  1. Nice article!

    I would have been interested to see how goal difference in the 1st half of the season correlated to goal difference in the 2nd half.
    I guess it would have followed a similar pattern to TSR but with the R values being a bit lower?

  2. Hi Tictac,

    I'll perhaps post on it.

    IIRC GD1 to GD2 has higher r than TSR to GD2 for same dataset. Weighted shots comfortably beats TSR for all leagues or subsets of leagues that I've looked at so far.


  3. Thanks for the analysis. It is important to be aware of the fact that many high correlations can be driven only by the best teams.
    In this context, the results also show us that mid-table or bottom-half teams simply don't sustain their performance as well as top-half teams: If they have been good for one season, their best players and the manager might leave them, and if they have been not so good, they will try everything to improve. This might be especially true for lower divisions, where the fluctuation of players is even more significant. And more fluctuation on the personal section leads, logically, to more fluctuation of results.
    Thus, I'd state that the performance of weaker teams is generally less predictable by their pure results from the past (not only by TSR). For better predictions, you need probably more concrete data (or other kind of knowledge) about players and the manager.

    Other point: Couldn't it be useful to use Spearman's rank correlation instead of Pearson's product-moment correlation? Normally, rank correlations are less dependent of outliers. For example, in the first plot, the "Old Firm" points are more than 20 goals above the rest, which explains there big influence on r. But if you express it in ranks, Celtic and the Rangers are only few ranks ahead of the other teams, as it is impossible to leave one or even more ranks between you and the following club. That means that the very best values should not dominate the correlations too much anymore.
    In my opinion, this could be a good compromise between not considering the best teams at all (r=0.14) and over-considering them (r=0.73), although I don’t know the r of the Spearman correlation.

  4. Hi Mark. The fundamental point you are making (as I understand it) is a very good one. Correlations between different metrics can be inflated when the variance in skill within a league is highly skewed, as it is in the Scottish Premiership.

    However, I think you have underestimated the value of TSR as a predictor of future GD in Scottish football.

    Both TSR and PDO are independently related to GD in the Scottish premiership, together explaining > 90% of the variance in GD in any given season.

    Since TSR and PDO are weakly correlated with each other, but strongly correlated with GD, you have to take into account PDO when you look at the relationship between TSR and GD.

    One way to do this is to use OLS regression with GD as the response and PDO as the sole predictor (GD2 ~ PDO2). The regression residuals can be used as PDO-independent measure of GD instead of raw GD.

    If you take this approach, you will find that early season TSR1 is a good predictor of late season PDO-independent GD2, at least in the Scottish premiership last season.

    I am not saying TSR is better than alternative metrics based on weighted shots or expected goals, just that your analysis does not accurately reflect the full potential of TSR as a predictor of future performance.

    Thanks for your consideration.


  5. Cheers Seth, that's great.

    the point I was trying to make was TSR being increasingly used in isolation as a stand alone stat, especially as a Stoke supporter.

    The first post on this blog was about Stoke, poor TSR, therefore always being branded as lucky and relegation favourites, despite a near league leading shot and save conversion rate, which was a product of Pulis' style of play.

    You had to actually watch them play a few times, especially at home. which admittedly wasn't everyone's idea of fun.

    cheers, great response.