Predicting the Rare and Significant from the Incidental and Commonplace.

The transfer window remains open and the net is awash with statistical comparisons between present incumbents and perceived replacements, but with a few notable exceptions, much of this fevered speculation fails to address the problems of sample size in the data used. What you see, especially over a single season is what you got, not necessarily what you'll get. However, even when regression towards the mean is addressed, a much more fundamental issue often remains, namely does the statistic used actually bear any relationship to the ultimate, desired outcome. Do the numbers we are attaching to a player, correlate and hopefully have a causative link with winning ?

Simply because a number is recorded, it doesn't naturally follow that the stat is valuable in evaluating a player, especially in the absence of context, such as the playing style of the team where he has amassed those figures. For example, interceptions are probably a skill. But generally the more interceptions a team makes, the worse they do in the game.

More interceptions (leading to worse results) should hardly be a great selling point for a player. It would be much better to try to identify how situations develop that require an interception to be made and purchase a player whom might prevent those situations from arising in the first place. That might be the route more successful, but less intercepting prone teams have taken.

The great interceptor may look good on paper, but you may be purchasing the equivalent of a signing who is really good at picking the ball out of his own net. Once you require his particular skills, the team has already dug itself into a substantial hole.

So it makes more sense to concentrate on numbers that at least positively correlate to success, rather than throwing everything at the analysis. The choices of these potential killer stats are readily available. For a small monthly fee, a few sites provide reams of individual player stats, sorted on a game by game basis and free, but less exhaustive data can also easily be found. Added context, such as pitch location divided by thirds isn't ideal, but it does exist.

One fairly obscure recorded stat is possession won in the attacking third, essentially a turnover without the opposition threatening your goal. In the NFL this type of turnover is gold dust and a positive differential during a single game in this statistics makes victory extremely likely. Such powerful events are relatively rare in gridiron and so over a mere 16 game regular season, we are unlikely to be able to readily distinguish the lucky from the good.

This particular problem can be eased by looking for a particular skill that is likely to be connected to this rarer, but valuable skill, but occurs in much larger quantities. Poorer pass completion rates for quarterbacks correlate well with the undesirable "talent" of throwing a less common and drive ending, interception and on the defensive side of the ball, the ability to defend a pass is a good indicator of the rarer event of catching an interception. We'll return to this later.

The Premiership equivalent of winning possession deep in opponent's territory appears to share both the rarity and partly, the worth of the NFL counterpart. To take Everton over the previous two seasons as an example, the correlation between final 3rd turnover differential and a positive match day outcome is significant and strong, despite the relative rarity of the event.

Over the last two seasons, Everton has averaged 2.6 such events per game and allowed their opponents to gain an average of 1.5 turnovers in the Toffees defensive 3rd. Above I've plotted the line of best fit that relates match day final 3rd possession turnover differential to outcome. The red bar represents the likelihood that Everton won the game, whilst the green bar incorporates the draw as well.

Everton bridge the gap between Champions League regulars and the remainder of the Premiership, so they are good enough to overcome a final 3rd possession winning differential of minus 4 and still  be more likely than not to take something from the match. Once their opponents enjoyed a positive differential of around eight or more, the Toffees' chance of a win drops below 10%.

By contrast, the outcome of tussles for possession in the middle third are much more numerous, but this particular differential in possession turnover provides virtually no correlation with match result. Despite the added opportunities for the better teams to demonstrate their skill and drive a gap in the respective successes posted by themselves and their opponents, it is rarer events, slightly closer to the defended goal that appear to determine where the spoils are most likely to go.

Post game knowledge of the differential between each side's possession steals in the middle third gives you absolutely no extra information as to the likely winner of a match involving Everton since August 2011. Simply picking the generic win % is likely to be as close to predicting the outcome of a group of otherwise anonymous matches as trying to glean information from the middle 3rd possession differentials from each game.

So we have more numerous, but apparently non predictive, middle 3rd possession interchanges and rare, but indicative final third events. Therefore, recruiting a player with final 3rd capabilities would appear to be desirable because of the strong correlation of that stat to a positive match outcome. Unfortunately, few players reach double figures for successfully winning possession in the final 3rd, even over two seasons of regular playing time.

To solve this inconvenience in trying to accurately establish players of final 3rd potential, we need to briefly return to the NFL. Interceptions, primarily by cornerbacks and safeties are rare, but defended passes, whereby the ball is knocked away from the intended receiver, much less so. The skillsets needed to perform both tasks is very similar. Good hand, eye coordination and speed, coupled with athleticism. So it shouldn't be surprising that defended passes are strongly correlated to full bloodied interceptions.

And the same is true of a soccer player winning the ball in the middle and final third. The skills required are virtually identical and the two stats are extremely strongly correlated with each other. Therefore, instead of trying to work with extremely small, often single figure sample sizes from the final 3rd, we can use the strongly related, middle third possession figures as a proxy for the trait that appears to drive match results, but turns up barely twice a match for a team of up to 13 outfielders.

How Winning Final 3rd Possession Relates to Winning Middle 3rd Possession, Everton 2011-13.

Playing Position.	Number of Successful Final 3rd Events per Middle 3rd Events.
Defenders.	5 per 100.
Midfielders.	10 per 100.
Strikers.	10 per 30.

Now instead of drawing information from events that in the case of Everton occur only five times every two matches, we are in a position to take a player's seasonal statistics from correlated events that happen over 20 times a match for each team. Winning possession in the middle 3rd may not correlate to winning, but it is tied intimately to the ability to win possession in the final 3rd..... and that stat does correlate to match success.

Choosing between two players with similar playing time, whom have both regained possession just three times in the final third, becomes slightly easier if we also know that one has also recorded 60 similar actions in the middle third compared to just 20 for the second player. The 60/20 middle third split gives the former a potential edge over the latter that is absent in the tied single figure sample of the stat that has really attracted our scouting interest.

Whether or not Everton's final 3rd possession winning stats are an integral part of just their match winning makeup, a spurious correlation or a strong Premiership wide theme, isn't the most important part of this post. Sometimes if you need to estimate a rare, yet valuable occurrence, it is better to look elsewhere at a more mundane, apparently unimportant, but highly correlated and much more commonplace alternative. That way you don't have to wait ten years until a potential transfer target bulks out his cv with rare and valuable actions.  

Unpeeling The Shooting Onion.

The quality of a foot shot in football is decided by a variety of parameters. This list, not necessarily an exhaustive one, includes shot location, the power of the attempt, the placement of the shot and the defensive pressure on the shooter. Most of these listed can, at least in some general way be recorded and quantified to allow further analysis. However, other possibly significant factors, such as pitch condition, player fatigue and quality of the previous pass are less easy to tabulate.

It is easy to overestimate the amount of control a player has over the execution of a shot. Variables, such as defensive pressure present an obvious and changing irritant for the shooter, but constants within the context of a single shot, such as shot distance may also apply some influence on the type and hence the quality of the attempt.

For example, distance shooting often requires a more powerfully struck shot, not only to make blocks by outfielders less likely, but also to get the ball to it's target quick enough to prevent the keeper from correctly positioning himself. But the requirement for greater power may relieve the player of some control of other aspects of the shot, such as placement.

Given sufficient time a keeper's movement and positioning allows him or her to cover the entire goal. But shots taken closer to goal may be hit with less power, but arrive quick enough at an area of the goal that the keeper is unable to reach in the time available. Therefore, distance shooting may require power to hurry them on their way and closer efforts may rely more on guile and craft to consistently direct the ball to it's intended target.

I've plotted the correlations between these more commonly charted shot characteristics and various shot outcomes. We may then try to build up clearer picture of how they are likely to relate to outcomes at the sharp end of the effort and to allow speculation as to whether the shooter is controlling the relationship or if the initial inputs are at least partly hard baking the outcome.

Shot strength is broadly divided into powerfully, normally and weakly hit shots and the three types show a gradual, significant trend as we get closer to goal. The perpendicular distance from the goal in yards is used to denote the x,y coordinate of each attempt.

Most shots are hit with normal power (obviously, from the name alone), but the percentage increases as we near the goal. The decline as we enter the box is keenly felt in the proportion of strongly hit shots. Over a quarter of shots are given a real thump from 30 yards, but less than 5% from the edge of the six yards box. Are we seeing the need to get the ball quicker to the target being proportionally more important from distance being displayed in the data ? Closer in, placement appears to become more important and presumably profitable than raw power.

The  interesting category is weakly hit shots, surely never an advantage for the ball striker. The steady increase in the proportion of weak shots as we approach the goal is possibly due to the higher intensity of defensive pressure or the eagerness to get a shot away from close range regardless of quality. Miscues will always happen, but something appears to be encouraging them in the six yard box.

The next graph is essentially a "high or low" plot for shot placement dependent on the strength of the shot. Shots from distance that are on target tend to be more likely to hit the goal plane vertically above the half way point. Possibly another manifestation of distance shooting being more powerful and tending to go high compared to placement orientated closer efforts.

If we imagine a goal kick as an extreme variation of the long distance shot, it is hit with power (to prevent opponents blocking the effort), it is hit aerially (again to reduce blocking, but also to carry the required distance, maximum range is achieved with a trajectory of 45 degrees to the horizontal). By contrast a pass or a short goal kick is hit with just enough power and more placement to avoid an opponent, but safely reach it's destination.

Long distance shots maybe the goalkicks of the shooting world and shorter ranged shots are the passes

We can further refine the shooting tendencies by looking at the preferred destination for on target efforts in regards to corners of the net, either top or bottom, again sorted by shot distance. Shots from distance are nearly twice as likely to arrow towards the top corner than the bottom. A controlled effort by the striker to try to hit the most difficult area for the keeper to reach or merely the natural outcome of the relative inability to keep powerfully struck, long range efforts from flying to an elevated height ?

The final graph looks at the sacrifice in accuracy you give up by hitting a more powerful shot compared to a normal one. Powerful shots are more likely to miss or be blocked than weaker, or normally hit efforts.

To summarize, from distance (30 yards), powerfully struck shots are at their peak, they take the aerial route more often, hit the top rather than the bottom corners of the goal, but lack the "on target" accuracy of normally struck efforts. A player can certainly chose how hard he hits a shot, but that the attempts then more frequently take to the air may be at least partly down to the difficulty in controlling the flight of a powerful shot, rather than conscious aim taking.

Once we home in on the penalty box, the areas of the goal that a keeper cannot cover adequately in the timescale of a normal shot rapidly increases. Hitting top corners or going for power is no longer a priority if "passing" the ball into an area of the goal that a keeper's angles have left exposed carries a higher likelihood of success.

Shooting appears to be a constant interplay of trade offs between power, placement and a multitude of other variables, with distance partly dictating the required approach and then possibly having a say in the most likely end point of the effort. It is easier to manufacture a shot from distance, but the skillset and rewards may differ from those needed from closer to goal and it is easy to speculate that the spread of talent for players taking shots may well differ at different distances from goal. Are there more Bale's than goal poaching Lineker's in the general striking population.

The introduction of more variables certainly widen the scope for evaluating shooters. Ultimately though, at the moment, we don't even really know how much of the process is fully in control of the feet of the shooter.

Robin van Persie Really Isn't Afraid to Miss.

A year ago I wrote a couple of posts about the shooting characteristics of Robin van Persie under the general heading of "strikers shouldn't be afraid to miss". The data was from 2010/11, so the Dutchman was still an Arsenal legend, rather than a dangerous opponent from a championship seeking rival who makes a once a season visit to the Emirates stadium.

To summarize the post, when comparing the chances of a van Persie goal effort resulting in a score against location corrected efforts for the other main Arsenal shooters, the Dutchman was clearly superior. However, in terms of accuracy, it was his group of colleagues that were more likely to hit the spot and no matter where the shot originated from, van Persie was slightly more likely to miss the target completely.

I speculated that the rest of Arsenal's strikeforce was taking the mantra of "make the goalie work" to rigidly, taking less risks and more guaranteed applause by failing to shoot towards the less accessible areas of the goal for fear of missing. In short, van Persie's shot placement was more adventurous, more prone to missing the target completely, but ultimately more rewarding in terms of goals scored as a result of this efforts struck towards the periphery of the goal, generally being more difficult to save.

I intended to compare van Persie's shot placement to see if he was indeed working with finer margins, but instead the post became a general look at shot placement as well as shot power on the regularly seen location based shooting models. That post turned up last month as Power and Placement , so now is a good time to revisit the original van Persie venture.

Placement is an obvious factor in determining the outcome of a shot, although the effect of shot power is likely to have a much greater influence on the final result. From data used in the previous link, shots hit from identical pitch positions, but aimed closer to the frame of the goal, increase the likelihood of scoring. But when going from a weak to a more normally paced shot from the same spot, the improvement in expectation is generally many times greater. Shot pace is likely to be a major factor in calculating goal expectancy, but it just isn't generally available, so we will have to include placement alone, largely ignorant of the potential effect of shot power.

Although both keepers and strikers have a "natural" side, it's unlikely, given the numerous body shapes a player has to adapt to get a shot away that one top corner is very much different from the other. Therefore, to see how van Persie's shot placement may have enhanced his performance compared to a shot location based model alone I summed his total top corner, bottom corner and central attempts and saw how they compared in terms of frequency to those of similar contemporaries, namely the top scorers from all the other Premiership sides from 2008 onwards and his fellow Arsenal strikers over the same time scale. This approach also boosts sample size.

Percentage of Shots Hitting Particular Areas of the Goal, 2008/13.

Player/Team.	Top Corners.	Lower Corners.	Top Central.	Lower Central.
R. van Persie.	15%	54%	8%	23%
Arsenal exc. RvP.	12%	50%	7%	31%
League Ave. Top Scorers	13%	50%	8%	29%

The impressions from the original post appear to be confirmed by more granular data. Robin van Persie does appear to find the top and bottom corners proportionally more often than his now former goalscoring teammates, as well as beating the pooled average of the top scorer from each permanent member of the Premiership club since 2008. Only Villa's cumulative pool of leading scorers has bested van Persie's ability to hit the corners over the last five completed seasons. For those fond of anecdotal evidence, the Dutch striker has also hit the woodwork 17 times in the last two seasons, perhaps indicating that he is likely to be shooting with an extremely narrow margin between probable glory and individual shooting failure.

By daring to miss, the premise of the original post appears sound, RvP seems to be boosting his scoring rate at the expense of his accuracy.

van Persie outperforms a simply model based around just shot location. If we then incorporate shot placement into an updated version and van Persie (or any other player) consistently outperforms that version, we need to entertain the idea of including other variables, such as shot power. And then onto defensive pressure or player movement and quality of assist. Models always fail to include every possible factor, but as long as we know what is included, they remain valuable. What's absent is the likely cause of discrepancies and this can lead to fruitful, further investigation.

Perfecting shooting models is an onion that's going to take some peeling.