How Game States Alter Chance Conversion Rates.

Ideally, if you are attempting to quantify an identifiable skill in a sport such as football, you would like both the conditions of the trial and the context within the game to be controlled. Penalty kicks fulfill many of these conditions. A free kick from 12 yards, taken at relative leisure without the intervention of defenders, where only the identity of a similarly skilled goalkeeper alters, is as good as it gets in football. Unfortunately, it is also a rare event and therefore as a way to differentiate a repeatable talent, it ultimately fails.

Shots from open play are much more common events, both from an individual and team perspective. However, the advantages of consistency of each trial that was present in penalty kicks is largely lost. A two yard tap in or a thirty yard volley each appear as an indistinguishable "shot" when all attempts are simply lumped together. 

On a team basis, the two polar extremes for goal attempts from recent seasons are Stoke, at their set piece dependent best (or worst) and optimistic, long range shooting QPR. Both side's struggled for goals, but measured by raw shots alone, Stoke appear the more efficient of the pair. In 2010/11 their conversion rate of 11% hovered around the league average over the last decade, while in comparison, QPR in their relegation season recorded conversion rates of barely half that.

However, the comparison is misleading, City's average shooting distance was just past the penalty spot and QPR's was very nearly at the edge of the box and also a couple of yards wider. Rangers can be faulted for shooting so regularly from distance compared to both Stoke and the rest of the EPL, but it is that misguided optimism that led to an apparently abysmal conversion rate. When shot position is accounted for both QPR and Stoke were converting the chances they elected to take with similar levels of ability. 

If QPR had elected to try to create chances closer to goal, their conversion rate on a shot by shot basis would likely improve, with no real change in shooting ability. Similarly if Stoke shot more from distance, their rate would likely fall, again with no requirement for an underlying change in talent. Tactical approach, rather than changing talent or masses of randomness can be a huge factor in fluctuating shot conversion rates. 

The disconnect between raw counting conversion rates and x,y based rates is obvious in the case of Stoke and QPR, but similar effects are present for all sides.

If we start by looking at the rate at which teams from the EPL have converted shots, regardless of any additional information such as shooting distance, there is a relationship of sorts between conversion rates in season one and those recorded in the subsequent season. The line of best fit appears to indicate that poor conversion rates in one season tend to be followed by poor, if generally slightly improved rates in the next season. At the opposite extreme, a side converting at a well above average 18% would on average fall to around 14% next term. 
So there is evidence of a difference in finishing ability between sides, but also a degree of regression towards the mean, implying an expected amount of randomness, also.
The case of Stoke and QPR's different shooting profiles illustrates that shot position is a major factor in determining a fair expected conversion rate for a side. Shot position is mostly a choice determined by the attacking side, but in some cases a side is also partly forced into shooting from greater distance as time expires from a disadvantageous scoreline position. Such situations when, but not exclusively, a side trails is often accompanied by their opponents in addition presenting a more defensive shell. 
More speculative shots, against packed defenses, intuitively is going to depress conversion rates. So again we have a situation where any side can find itself in a situation where the trials commonly used to calculate the strength of season on season correlations between conversion rates are being altered by circumstances that are partly out of control of the attacking unit. In short, if your defense, through a combination of random chance or poor play puts a side consistently in poor game states, then your shooting conversion rate is likely to fall through poorer quality and better defended chances arising at the opposite end of the field.

We can see possible evidence for more frequent shooting going hand in hand with less efficient conversion rates by plotting Arsenal's total shot numbers and their seasonal conversion rate from 2002-2003 to the present. Random chance inevitably will play a part in the Gunners grabbing or conceding the opening goal, but how frequently they found themselves in either a good or bad game state will then alter the quality of the subsequent shooting trials. Around three quarters of the sides which have played for five or more seasons in the EPL since 2002-03, exhibit the same trait of decreased efficiency with increased shot frequency.

Arsenal, along with the other big four sides, tends to have the simplest game states. Leading is always good, but such is their quality, that drawing and obviously losing is invariably bad. Therefore, the average game state they experienced over a game or a whole season often corresponds closely to the amount of time they spent winning, drawing or losing. This allows us to express a good proxy for game state in a single number by using the proportion of time spent leading, as well as giving half the weight to time spent drawing over the period of a single game or a whole season. 
And the same pattern is seen. The poorer the average season long game state experienced by Arsenal, the more shots they had. Similarly, for Stoke, a side which have a more ambiguous relationship than Arsenal with a stalemate (sometimes against weaker sides it represents a poor game state, more often though, against better teams, it is a good one). In all matches where they had a better than average game state, they took just 8 shots per game, compared to an average of 12 when it was below average.
So we have a connection between more, less efficient shots being taken in poorer game states and while the former may partly drive the latter, the changing game state also alters, for better or worse the likely conditions of the shooting opportunities. Either in the longterm, depressing an already (partly luck driven) poor efficiency or enhancing an already impressive one.
In short, the context of game states is likely to have a significant effect on conversion rates and may even act as a decent proxy for shot distance and defensive pressure.
There are no short cuts to calculating game states. Final scorelines can mislead, a side can trail for 85 minutes and then grab two late goals, or score twice early and concede in second half injury time. Two 2-1 wins, but with vastly differing game states and in all likelihood, dissimilar goal attempt profiles. 
Time spent leading/drawing and losing are the building blocks, but then we have to decide how happy to defend or eager to attack each side will be in the commonly occurring stalemated scoreline. So we also require an estimate of team quality to further quantify game state in this all encompassing area of analysis.

To demonstrate how game state alters the conversion rates of a side from one season to the next, when squad turnover is likely to be light, above I've plotted paired conversion rates from consecutive seasons for Arsenal, again since 2002-13. For amalgamated data comprising 38 games in each point, the correlation is disappointingly poor. The temptation is to assign the lack of correlation entirely to random variation, and while that undoubtedly exists, we also have a naive model, lacking in detail. 
If a side has the good fortune to lead lots of games and if their style of play allows, they can sit deep, sit on their likely high conversion rate and attack their opponents on the counter, where chances may be fewer, but they will likely be of much better quality because their opponents are actively seeking to pull goals back. If during the next season, they fall behind more frequently (possibly because of a poorer defence and/or an unlucky attack), they could easily find themselves with a much reduced conversion rate, as they are forced to trial their shooting skills against more densely packed and better organised defences. 
The poor season on season correlation could be down to a combination of randomness, but also seasonal variation in game state.

If we wish to know how conversion rates correlate from one year to the next, looked at through the lens of total shots, we should at least try to accommodate important factors that appear to contribute, such as game state, both previously and in the season in question. So, instead of plotting paired conversion rates, I've taken the conversion rate in the previous season, along with the game state from that year and the game state experienced by the side in the subsequent year and projected a conversion rate for that subsequent campaign using these three factors. 
In short, if team A (appropriately Arsenal) convert at a certain rate under x average game state, what will they do under y average game state with mostly the same squad based on previous patterns. I've plotted this projection against the actual conversion rates above and the r^2 jumps to nearly 70%.
Game states and previous conversion rates go a long way to explaining, why a side records such apparently random conversion rates in consecutive years. Randomness exists, but other more concrete causes are equally as important.