Consistent or Calamitous: How good is your goalie?



David De Gea won the match for Man United at the Emirates last month. Under siege for much of the second half, de Gea’s brilliant performance preserved United’s lead at a crucial stage of the match. He made a total of 14 saves in the match, equaling the EPL record set by Tim Krul for Newcastle in 2013.

How many goals does a high-quality goalkeeper like de Gea save his team over the course of a season? Pundits frequently claim that a goalkeeper is worth so many goals or points, but this has never actually been measured. In this post I take advantage of a novel feature of Stratagem’s dataset to infer the number of goals that EPL goalkeepers have saved or cost their teams in the last two seasons. Who is bailing out their defence, and who is letting them down? Would United be a top-4 team without de Gea? Is either of Mignolet and Karius a decent shot-stopper?

The difficulty of a save


When a goalkeeper fails to make a routine save it is described as an error, costing the team a goal. But when he makes a superb fingertip stop he is thought to have saved his team a goal. To measure a goalkeeper’s shot-stopping ability, we must assess the difficulty of the shots that he faced.

A nice feature of Stratagem’s dataset is that it includes a subjective assessment of the quality of every shot, along with a record of the outcome (missed, saved, intercepted or goal). Each shot on goal is allocated a shot quality rating on a scale of 1-5, where a '1' represents a wild, scuffed or poorly hit shot (e.g. Lukaku’s recent overhead kick attempt against Leicester), and '5' an excellent shot that gave the keeper next to no chance (such as Alonso’s free kick goal against Spurs, or Ozil’s volleyed goal against Newcastle). A standard shot – one that tests the goalkeeper but that he should have a reasonable chance of saving – is rated as a '3' (think Firmino’s equalizer against Arsenal last month).

However, we can’t solely assess goalkeepers on the quality of the shots they face: the quality of the chance – before the shot was taken – must also be taken into account. After all, a shot from one yard out is likely to result in a goal irrespective of how well it is hit. Fortunately, this is exactly what expected goals (ExpG) measures.

I constructed a simple model to assess the probability of a goalkeeper making a save given the quality of the chance and the quality of the shot; details of the model are given in the appendix. For shot quality I used Stratagem’s rating, for chance quality I used an ExpG model I wrote about recently (accounting for the distance and angle to goal, the number of intervening defenders, and the defensive pressure exerted on the shot-taker).

Figure 1 shows the percentage of shots (on target) that were saved for a given shot quality rating, having controlled for the quality of the chance. The results are shown for three goalkeepers in the 2016/17 season: Heurelho Gomes (blue line), Artur Boruc (red line) and David de Gea (black line). The blue shaded region shows the results for the average EPL goalkeeper.

Figure 1: The proportion of shots -- on target, and not blocked by an intervening outfield player -- saved as a function of Stratagem’s shot quality rating (controlling for chance quality by fixing the ExpG of the chance to 0.1). The blue, red and black lines show the proportion of shots saved by Heurelho Gomes, Artur Boruc and David de Gea in the 2016/17 season. The blue shaded region shows the average over all EPL goalkeepers that season. 

The average EPL goalkeeper saved nearly 80% of standard shots (rated as a ‘3’) last season, and nearly 99% of poor shots (rated as a '1'). However, he saved only 65% of well-hit shots ('4'), and only 8% of excellent shots ('5'). Artur Boruc’s save ratio was almost identical to the EPL average, but David de Gea’s was clearly superior: while the average EPL keeper saved under 80% of standard shots, de Gea saved more than 90%. He did even better with shots scored as a ‘4’, saving 20% more than the average goalkeeper. However, Heurelho Gomes cost Watford goals last season. Although he was reliable in making routine saves (shots rated as '1' or '2'), he saved a significantly lower proportion of shots rated '3' or higher.

How many goals are goalkeepers worth?


Figure 2 shows the number of goals that each EPL goalkeeper saved or cost their team in the 2016/17 season (blue bars), and in the 2017/18 season (up to gameweek 22, red bars). It is calculated by comparing the actual number of goals conceded by each goalkeeper to the number of goals that the (hypothetical) average goalkeeper would have conceded from the same shots, based on the model described above. A negative total indicates that the player saved their team goals relative to the average goalkeeper; a positive total indicates that they cost their team goals. Only players that played at least 14 matches in either season are plotted.

Figure 2: The number of goals that each EPL goalkeeper saved (-ve) or cost (+ve) his team over the course of the 2016/17 season (blue bars) and 2017/18 season (up to gameweek 22, red bars), calculated by comparing the actual number of goals they conceded to the number the average goalkeeper would have conceded from the same shots. Only goalkeepers that played at least 14 league matches in either season are shown.

David de Gea has been the best performing goalkeeper in the last two seasons, saving Man United around 8 goals last season and nearly 10 so far this season. Without him it is highly likely that United would not currently be in the top four. Burnley goalkeepers Tom Heaton (16/17) and Nick Pope (17/18) have both performed extremely well, too: is this somehow a consequence of the way Burnley play, did they both happen to be in very good form, or are they genuinely very good goalkeepers? It will be interesting to follow their respective careers over the next few seasons.

Petr Cech and Hugo Lloris both saved their teams a significant number of goals last season, but they appear to be underperforming this time around. I wonder if Arsenal and Spurs fans would agree with this assessment? Similarly, would Swansea fans attest to a significant improvement in Fabianksi this season? 

Simon Mignolet has been decidedly average. With the exception of Claudio Bravo – who was replaced in the summer – he is the worst performing goalkeeper at a top-6 team over the last 18 months. Having confirmed that Loris Karius is now the first choice goalkeeper at Anfield, Jurgen Klopp finally appears to have lost patience with Mignolet. However, Karius also looks vulnerable, and his stats are not great either. Purchasing an elite goalkeeper should be very high on Liverpool's to-do list.

Heurelho Gomes has been the most costly goalkeeper over the last season and a half, conceding nearly 12 goals more than the average goalkeeper last season, and 5 more this season. I haven’t noticed Watford fans complaining about him though, so it would be interesting to know what they think of this. The only other goalkeepers to have cost their team more than five goals in a season are Joe Hart and Wayne Hennessey. 

Of course, comparing goalkeepers by the saves they make neglects other important aspects of their job. Collecting crosses and accurate distribution are also crucial parts of their skillset. However, as with a striker’s link play, or ability to create space, these actions are much harder to quantify. 

Are Goalkeepers undervalued in the transfer market?


Relative to the expected performance of the average EPL goalkeeper, David de Gea has saved Man United nearly 10 goals so far this season. That's equal to the total number of goals that Lukaku and Morata have scored, which is not even a fair comparison: we should really compare de Gea with the number of goals each striker has scored relative to the number an average striker would have scored in their place. By that metric, de Gea could well be the highest-performing player this season – in any position. 

Yet top goalkeepers appear to be priced significantly lower than their teammates in the transfer market. David de Gea does not feature in the top 100 players by transfer value calculated by the CIES Football Observatory. Last season he was ranked 58, six places below Jamie Vardy. Only two goalkeepers made it into the top-50 (Courtois and Oblak), and only ter Stegen does this year. Just one goalkeeper appears in the list of the top-50 most expensive transfers of all time. Even in the Fantasy Premier League, de Gea has accumulated more points than any striker bar Kane this season, yet is worth less than half his value in the game.

Goalkeepers play a unique and highly specialized role in a team, yet they are consistently undervalued. This could be one of the largest inefficiencies in football. As my evidence shows, de Gea could turn out to be the decisive factor in Manchester United securing Champions League next season. Conversely, a lack of an elite goalkeeper could see Liverpool falling short. 


Many thanks to David Shaw for reading and commenting on multiple version of this post.


This article was written with the aid of StrataData, which is property of Stratagem Technologies. StrataData powers the StrataBet Sports Trading Platform, in addition to StrataTips.



APPENDIX 1: Save Probability Model

The logistic model for save probability is defined as:

SaveProbability = 1 -  1 / ( 1 + exp⁡(-y)  ) ,

where,

y = β0 + β1 * ChanceQuality + β2 * ShotQuality .

ChanceQuality is the ExpG score for the opportunity, determined using the ‘extended’ model described here. ShotQuality is Stratagem’s shot quality rating for the shot, rated on a scale of 1 (poor) to 5 (excellent). 

The β coefficients were determined using logistic regression. For individual goalkeepers they were obtained using the shots on target faced in a given season; for the average EPL goalkeeper all shots faced by goalkeepers that played at least 14 matches were used. The coefficients for the average goalkeeper in the 2016/17 season are β0=-7.56±0.30, β1=-8.62±0.41 and β2=1.81±0.09

Comments

Post a Comment

Popular posts from this blog

Using Data to Analyse Team Formations

Image
I recently presented a research paper at the FC Barcelona Sports Analytics Summit on detecting and analysing team formations using tracking data (click here and scroll down to find the paper). I thought it would be nice to publish a shorter version on my blog. The work, which was awarded the "best research paper" prize at the conference, was done in collaboration with Mark Glickman at the Harvard Sports Analytics Lab. Too long? You can take a look at the poster version here , or the cartoon version here . Introduction A vital aspect of a football manager’s job is to select team formations – the spatial configuration of the players on the field. The choice of formation determines player roles, how they interact, and influences the playing style of both teams during a match. Despite their central role in team strategy, descriptions of formations are largely reliant on classifications based on the number of defenders, midfielders and forwards: crude summaries of pla...
Read more

Structure in football: putting formations into context

Image
This article was published as a chapter in the FC Barcelona Innovation Hub Football Analytics Guide, 2020. The Guide is freely available for download by filling out the online form here . Links to other work cited in this chapter are given at the end. Formations are the foundations of tactics in football. They provide structure to a team that helps the players to position themselves and broadly defines their specific roles in attack and defence. They are the means through which managers attempt to maintain control of high-value territory while denying their opponents the same.  The debate around formation tactics is as old as the game itself and is a central theme in the history of how the game is played. Innovations in formations have mirrored the evolving balance between defensive solidarity and attacking flair, discipline and freedom of expression and outcomes versus entertainment. From the early 2-3-5 ‘pyramid’ in the late 1800s to the more balanced W-M formation of Herber...
Read more

From Sessegnon to Sanchez: How to calculate the correct market salary for EPL players.

Image
In the 2018 January transfer window, Manchester United signed Alexis Sanchez on a four-and-a-half year contract. He had just turned 29, which is  towards the end of the peak years for an elite forward. Yet Ed Woodward made him the highest paid player in the country, offering him a reported £350,000 per week , which equates to £18.2 million per year, or £82 million over the duration of the contract (before bonuses and signing-on fee). United have undoubtedly received a spectacularly poor return on their acquisition. Granted, Sanchez has had his fair share of injuries, but it is clear that United chose to massively overpay a player whose abilities were likely to decline. Putting their recruitment issues aside (for now), what would have been a more reasonable salary for an EPL player of Sanchez's age and calibre? What should Woodward have offered him? Market value salaries We can attempt to answer this question by combining two very useful datasets: the Transfermarkt.com ...
Read more