The majority of the football based research has been focused quite naturally on the English Premier League, most notably by James Grayson, who demonstrates both the repeatability and predictive nature of the stat for the EPL.
Very little of the detailed validation that James has undertaken for the EPL has been done on any other the other world leagues, but this hasn't prevented a combination of a respectable TSR and a relatively low league position earning a non EPL side team an "unlucky" tag.
If we first consider the case of possession, It has largely been discarded as credible measure of team strength. Barcelona's use of the tactic, with varying degrees of success, suffered at the feet of Bayern Munich in the 2013 ECL semi final.
Spain's finest held the majority of the ball, but fell to a 7-0 two legged thrashing. And in more humble surroundings, Stoke mostly under Pulis, graciously allowed themselves to be continually "dominated" on all fronts, even at home, but still won enough games to safely avoid the drop each season.
Stoke Shrug & Say No to TSR or Possession Stats. |
It is only when the top three or four teams are removed from the plot that the correlation for the remaining teams becomes largely random.
The constant presence of a handful of sides which dominate in a league, not only in wins, but in possession and shots by simply outclassing the majority of their opponents, therefore, may potentially create a misleading correlation, where only a weak one exists for the majority of teams.
So might the illusion of possession being generally correlated to results be repeated to some degree, in some leagues for TSR.
The Scottish Premiership has largely been a two horse race between Rangers (until their demotion following their liquidation) and Celtic. The league has an unusual format, comprising of 12 sides and the table splits in two once each team has played three matches against each of their rivals, enabling a 38 game season to be played out.
In common with the EPL, a Scottish Premiership side's TSR in the first half of the season appears to show a strong correlation to goal difference in the remainder. Therefore, TSR appears to do a good job of predicting a useful future quality of teams in Scotland's top flight.
The coefficient of correlation is a very healthy 0.71 for the four seasons from 2010/11 onwards. However, the scatterplot is fairly lopsided, there is very nearly daylight between the six points in the top right hand corner, four Celtic seasons and the only two from Rangers, and the rest.
As with possession, where a strong correlation is inferred because of the constant presence of atypical sides which partly outclass the rest, have Celtic and Rangers exaggerated the implied predictive power of TSR for the remainder of the teams, outside of the "Old Firm" by helping to create an impressive r value?
If we crudely remove the six "Old Firm" points from the plot, the scatter plot and the coefficient of correlation between past TSR and future goal difference for the remaining ten per season Scottish sides alters dramatically.
If we remove Rangers and Celtic, r falls to 0.14 as does the predictive power of the relationship. TSR may still tell us something about the remainder of the season for these non elite sides, but without a highly likely combination of a regular, out shot, defeat by either Celtic or Rangers, our conclusions become much less dramatic.
The separation of abilities in the EPL is perhaps less stark than in Scotland's top flight, but a division along the lines of Simon Gleave's "Superior seven and threatened thirteen" has largely existed over the last decade.
Once again, the coefficient of correlation is high if we include all Premiership sides and although the clustering of teams at the middle and bottom isn't as pronounced as in Scotland, there does appear to be an effect.
If we remove every game involving at least one of Simon's superior seven, we get a truncated season in which the threatened thirteen only contest home and away matches against one of the twelve remaining teams from this group.
If we now split the season in two and regress a threatened thirteen side's TSR in the first half of the season to their goal difference in the remainder of the season, the coefficient of correlation again falls. This time from 0.73 to 0.29.
Admittedly, the sample size has also fallen in both cases when the very best sides are removed, but the confidence with which TSR is used indiscriminately across and perhaps within leagues to evaluate certain types of sides perhaps should be questioned.
Just as the past possession record of sides is largely irrelevant to many future outcomes, (even when we can easily produce a scatter plot between possession and outcome to create the illusion of a strong, universal relationship), previous team TSR might also be much less of a factor in the prediction of some types of games.
In conclusion, we seem to get high r values for the relationship between TSR at half season and future goal difference in the remaining games, in leagues where there are a couple of consistently dominant teams, in every aspect of play.
Barca and Real Madrid in Spain, Bayern Munich in Germany, Celtic and a soon to be reunited Rangers in Scotland, the superior seven in the EPL.
The coefficient of correlation for the previous four full seasons, for EPL, Spain, Germany and Scotland are, respectively 0.77, 0.73 0.68 and 0.71.
In France, where dominant sides are less in number and also less dominant, r is 0.51, in Italy it is 0.53.
And most pertinently perhaps, in the bottom tier of the English football league, where runaway winners are rare, r for TSR to future goal difference from the mid season point over the last four seasons is barely 0.4.
So at the very least, the clear assertion that TSR is a good indicator of likely future performance, in every league and within every league at every level, ranging from the English Championship to the MLS and the Egyptian league to the Australian professional league, should also be backed up by the kind of thorough and rigorous validation that James has carried out for the EPL.
Without that, the strength of any possible correlation can only be guessed at.