Friday, 27 February 2015

"Penalty Kick to Chelsea!"

The use of data to inform and illuminate is gradually creeping into the mainstream media with varying degrees of success. The best examples legitimately combine excellent writing, an engaging narrative and data collection and interpretation, to produce a finished item that can be enjoyed and debated on many levels.

The Guardian's Sean Ingle currently sets the bar at a high level.

The most recent example sees Sports Writer of the Year, Martin Samuel adding a dash of data to his piece on Chelsea's penalty record in the Premier League in the Mail Online.

Mourinho has complained loudly that his team are hard done by when penalties have failed to be awarded this season and Samuel cites statistical analysis carried out by Opta to support Jose's assertions. (Although to be fair to Opta, they appear to have merely undertaken the collection part of the analysis and any interpretation of the data appears to be wholly the work of Samuel).

Samuel's approach follows a well trodden route. It uses Chelsea's rate of being awarded spot kicks in the Premier League this year (1 per 13 games in 26 matches) compared to their rate in the Champions League ( 1 per 1.8 games in 7 matches) and also against those of their English rivals from the CL and domestically.

Samuel then uses the "whopping additional 11.2 games needed to win a penalty in the Premier League" as the main "corroborative" evidence that Chelsea are getting a raw deal.

Firstly, referees appear to have particular regional traits, especially in areas such as on field discipline, where card make ups can vary considerably from one country to another. Therefore, is shouldn't be a surprise if Chelsea's CL refs, comprising Croatian, Spanish, Italian, Dutch Turkish, Norwegian and Swedish may differ in their interpretation of what constitutes a penalty kick compared to their Premier League counterparts.

Secondly, the level of competition faced by Chelsea, particularly in the CL group matches hasn't been very strong and more spot kicks tend to be conceded by teams which are vastly inferior to their opponent.

The Euro Club Index  rated the average of Chelsea's three group opponents at 2555 ranking points, the equivalent of a group made up of three teams of the average quality of a Stoke City. The easiest team from the group, NK Maribor currently rank 208th in the Index, inferior to virtually every current Premier League team.

(As an initial stage to the statistical analysis of Chelsea's seven game Champions League campaign, offered without any additional interpretation, two of Chelsea's four CL penalties came in their 6-0 defeat of Maribor).

But the main issue we should take with this mainstream article is in the use of headline penalty award rates derived from sample sizes that are unequal and limited in size.

Events do not happening in neat evenly spaced distributions. For example a fair coin which dutifully records a head followed by a tail should be treated with suspicion, rather than a confirmation that it is fulfilling an obligation to land a head or tail with equal likelihood.

Chelsea could easily have a typical Premier League likelihood of receiving penalty kicks of around one every 4.7 games and be awarded just two in 26 matches. The chances of this occurring or worse is around 7%. And the chances of any top side suffering this apparent injustice is greater still.

Relatively uncommon events will inevitably produce prolonged periods when they do not occur and others when they appear unnaturally clumped together, and this does not in itself provide evidence that a team is getting a raw deal.

You could have used a similar seven game Premier League sample to show have hard done to Manchester City were when they went into their 8th of the season match against Spurs without a penalty to their name.

90 minutes later, they had three.

Randomness is a much more likely candidate than conspiracies to explain Chelsea's record of penalty awards in 2014/15.

Saturday, 21 February 2015

Place Kicking and Six Nations Grand Slams.

In this previous post I looked at the effect of place kicking ability on the result of a single rugby union match based on rating kickers with a simple logistic regression model. The results highlighted the development of George Ford, England's current number 10, from his entry into the game with Leicester to his current position as an international kicker.

A more useful application of kicker ratings in rugby union is to compare the range of match outcomes over a competition, such as the Six Nations, bought about through the natural variation around a kicker's likely talent, as well as the effect of replacing that talent with an average value from the current pool of place kickers.

In this way we might usefully show the contribution made to a country's success by their regular kickers compared to that expected from an average replacement level of kicking ability.

The Grand Slam isn't a particularly rare event in the history of the Five and now Six Nations, it has been won in 40% of the tournaments since 1947 and was last won in 2012 by Wales.

With the exception of half a dozen kicks from Rhys Priestland, Leigh Halfpenny was the place kicker for each of Wales' five 2012 victories. England relied upon Owen Farrell, Scotland, Grieg Laidlaw and Ireland, Jonathan Sexton, while France mixed and matched between four different kickers and Italy used three different kickers.

I have used the kicking stats for each of the kickers from their previous season, both at club and international level. The majority of the Six Nations kickers had well in excess of 100 penalty or conversion attempts in that period and these kicks have been used to project how successful each kicker would expect to be when faced with the kicking opportunities during the Six Nations based on the difficulty of each kick.

I've then simulated the probabilistic outcome of all 153 kicks made during the competition and re scored all 15 matches by adding the outcome of simulated place kicks to those points scored from tries and open play kicks. Each "tournament" is replayed 10,000 times.

Wales won three of their five matches by a converted try or less, but even over 10,000 place kicking match simulations they remain the overwhelmingly most likely side to have taken the championship. Although the possibility that a single bout of unlucky place kicking could lead to a single defeat is clearly demonstrated by the likelihood of a Grand Slam falling to 73% compared to a 97% title chance.

Wales had by far the best points differential, which is used as a tie breaker if league points are equal and this advantage spills over into the simulations when they are tied with others on eight points.

There was a tiny possibility that England could have won the Grand Slam, although they were mostly competing with Ireland for second spot.

Ireland's comprehensive defeat by England effectively ended their chances of "winning" a virtual Slam, no amount of excellent Sexton place kicking could over turn the 10 points to 0 try scoring differential. But their heavy actual and potential defeat of Italy kept alive their chances of second spot in the simulations. This also indicates the large role played by the weaker sides in determining positions at the top of the final table under the current regulations.

Rhys Priestland began the tournament as Welsh place kicker and although statistically inferior to Halfpenny in the previous season he would expect to kick better than his poor, small sample efforts in 2012. Because of his poor start, Halfpenny remained Wales' primary place kicker during 2012.

In the plot above, I've simulated Wales' 2012 results firstly with Priestland and Halfpenny as the place kickers, each with the expected ability that they had demonstrated in 2011 for Wales and the Scarlets and the Blues respectively.

I've then re run the simulation using the modeled kicking ability of an average, regular kicker from the 2011 season to see the potential cost of playing with an average, rather than exceptional kicker, in the case of Halfpenny.

An average kicker costs Wales about two championships per 100 simulated trials, falling from around 97 with their actual choice of kickers to 95 with an average replacement. So Wales' 10 tries shared around each of their five games, bettered only by Ireland's 13 appears to make them worthy champions, almost regardless of kicking ability. However, Grand Slams, which are unforgiving of a single slip up, fall from by six per hundred from 73 to 67 with a par kicker.

Probably just as importantly, England win one Slam per 100 when Halfpenny takes the majority of Welsh kicks, but the agony/ecstasy is doubled to two if a par kicker takes the field.

Sunday, 15 February 2015

How Important Is Place Kicking in Rugby Union?

The importance of the boot in rugby union is increasingly evident. Not only as a means of gaining territory, but most obviously as a way of advancing the scoreboard through both penalties or conversions.

Place kicking is the most visible of set plays in rugby, occurring with relatively frequency in an otherwise fluid moving sport. It therefore, provides an ideal opportunity to model a repeatable trial, with few important variables and come up with a ranking table for kicking ability.

The position from where kicks are taken are obviously the most important factors, but the player's preferred kicking foot and the side of the pitch from where the kick was taken can also impact on the likely success rate. The model is described in more detail here.

Over half of the points scored by each team in England's recent 21-16 win over Wales in the opening match of the 2015 Six Nations came from the boot. England's George Ford contributing 11 of their 21 points (and potentially may have scored 16 place kicked points), while Wales' Leigh Halfpenny adding 8 points (potentially 11) to the three from a Dan Biggar drop goal for the beaten hosts.

Hands up if you're good enough to kick for Wales.
England crossed the tryline twice to Wales' sole touchdown, but the game remained sufficiently close for Brian Moore to describe Ford's 77 minute penalty as the most important kick of his career. The demand for instant sound bites contributes to such statements, but Ford's kicking career is defined by much more than a 40+yard attempt on a memorable night in Cardiff.

Ford's progression can be measured by comparing his kicking performance as a raw youngster for Leicester and England U20's  to his more recent efforts in the Premiership with Bath.

91 typical penalty attempts from the 2011/12 season from the young George Ford yielded 58 successes. Had these kicks at goal been taken by the more accomplished version of the fly half from the 2014/15 Premiership season, an extra 14 kicks might have been landed, on average.

Similarly, 54 successes from 67 Premiership kicking attempts so far this season would likely have yielded just 42 successes if they had been taken by a player with the kicking ability of the younger Ford.

It shouldn't be a surprise that Ford has improved as a kicker from an 18 to a 21 year old. And we can use the respective models to simulate the range and likelihood of outcomes for the recent Wales England match if the kicks were taken firstly by the younger Ford and then the more experienced incumbent of the number 10 shirt.

In short, we can judge the impact of a team playing with both an accomplished kicker and also a less skilled one, albeit simply through youth and inexperience.

Using the likelihood that Ford and Halfpenny are successful or not with all of their their respective kicks, based on both the difficulty of the attempts on opening day and each player's historical success rates, it is possible to simulate the possible final scorelines by adding these probabilistic outcomes to the points scored from tries and drop goals.

Unsurprisingly, England win an overwhelming 97% of the matches that pit the current kicking talents of Ford against Halfpenny. The latter is currently the world's best kicker, while Ford is a very good international place kicker, but had more opportunities on the night. In addition England outscored their host in points scored from non place kicks.

However, if we use Ford's inferior kicking stats from his stint as an England Under 20 international to create his model, England now win just below 90% of simulations, drawing 2%. More significantly when the final kick from the actual game has been simulated after 77 minutes, Wales are either leading or within three points of England in 34% of the simulations when England's place kicking is inferior, compared to just 10% of the time when Ford is modeled at his current talent level.

This potentially changes the dynamic of the final three minutes. A third of the time Wales are either defending a lead or need just a converted penalty to rescue something from the game.

It is tempting to define a players career by the outcome of high profile, pressure kicks. But in truth, England gained their win in Cardiff by recognising Ford's development from a promising 17 year old to a twenty something with excellent international place kicking abilities.

The fall in England's win probability from a near 100% to just over 90% when they are forced to rely on a weaker kicker, highlights that in the modern game of rugby, the highest quality of place kicking appears to be a luxury you must afford.