This article is part of a series that aims to re-think how we look at the NHL Draft.
The previous posts have looked at NHL draft market inefficiencies. These are various statistics that predict success outside of the draft position. In this article, we are going to do the opposite and show that a commonly held belief has no predictive power.
The belief in question? That you can excuse a prospect's low production level if he plays in a professional league ahead of his draft year because simply playing in a pro league as a teenager is such an impressive feat in and of itself. This analysis will show that this is not the case. Once scoring is accounted for league quality does not predict success.
Defining Expectations
To find which players have over/underperformed their draft position, we need a way to estimate the expected value of each prospect based on their draft position. This has been done in a previous article in this series which outlines the process behind this draft pick model (you can find that article here).
Note: We will be using Evolving Hockey's Goals Above Replacement (GAR) metric for this article. It's a catch-all metric that measures how much a player contributes to their team's performance. Higher GAR values represent more productive players.
The pick-value chart above gives us an expected value for each player based on their draft slot. Now, all we do is take the player's cumulative GAR in their first seven seasons and subtract off expectations and we have their GAR relative to expected.
For example, let's look at Nail Yakupov. Yakupov was picked No. 1 overall in the 2012 NHL Draft by the Edmonton Oilers. As a result, his expected GAR from the chart is about 65. Unfortunately, he only produced about 8 goals above replacement in the seven seasons after 2012.
As a result, Yakupov's value above expected is -57 (8-65). In other words, He dramatically underperformed his draft position. Yakupov is known as a generational draft bust, so a big negative number for him checks out.
Now all we have to do is apply this formula to every player drafted from 2007 to 2014, and start looking to see if any statistics can help us predict value above/below expectations.
The NHL draft market will have had all these statistics available to them, so if they are correlated with values above expected, it means the market was failing to account for them properly.
Does League Quality Matter?
For this exercise I will use Patrick Bacon's NHLe translations. The higher a league's translation factor, the more difficult the league is.
If league quality matters enough that you can excuse a prospect's low production just because he's playing in a tough competition, there will be a relationship between value above expected and the difficulty of a prospect's league.
This is important because we've already proven that low-scoring prospects underperform their draft position. So, for league quality to be a reasonable excuse for a prospect's low production, league quality will have to at least make up for that difference.
Testing this hypothesis is a little more difficult than previous ones because it's incredibly important to show how league quality moves independently of scoring. This is a problem because on its own, playing in a top-tier league is a good predictor of future success.
Here is the cumulative value above expected among players whose primary league in their draft year was either the KHL, SHL or Liiga compared to everyone else.
Not only did these professional-level players have higher upsides, but they had higher floors too. We can see this because they were also far more likely to simply out-perform their draft position.
Here a successful pick is just a player who produced more GAR than expected based on draft position, and an unsuccessful pick was a prospect who didn't.
Location, Location
So, by now you may believe my hypothesis was wrong. Maybe playing in a pro league is a signal of future success. This conclusion is problematic because league quality and prospect scoring tend to move in the same direction, especially outside of North America.
This correlation makes sense because European prospects have many professional and junior leagues close to where they live. So, the best prospects can move up and play in more difficult leagues. While North Americans can do this too (Auston Matthews did), it's far less common.
As a result, this analysis will test for trends in both the population as a whole and just among European skaters.
Accounting For Scoring
So, does league quality actually matter? To answer this question we are going to use a handful of regression models because regression analysis will help us see how league difficulty affects GAR above expected while holding scoring constant.
The models will be both continuous (linear) and binary (logistic). The continuous model looks at the player's value above expected as defined above. The binary models better account for outliers and define each player as a success (1) or failure (0). Successes produced more GAR than expected, failures produced less.
Model 1a, 1b, 2a and 2b show the same conclusion as above. On its own, there is a statistically significant relationship between league difficulty and prospect value above expected (Model 1a / 2a). League difficulty can predict both values above expected and success rates. This relationship becomes even stronger when only looking at European players (Model 1b / 2b). This makes sense given what we looked at above.
The interesting part is when we add scoring into the equation. once scoring is included, the coefficients on league difficulty become statistically insignificant.
This is true whether looking at both model types, plus with and without North American players. This combines something we already knew with a new insight. First, it reaffirms our previous belief scoring matters, even after accounting for draft position.
More interestingly it also shows us that once you know how much a prospect scores, whether or not he plays in a more or less difficult league contains no additional signal. This is true in the entire population and with North Americans filtered out.
So yes, prospects (especially Europeans) are far more likely to be good NHL players if they primarily played in a professional league in their draft year. They tend to score at incredibly high rates.
The problem is that once scoring has been accounted for, league quality fails to predict future success. So, that low-scoring prospect is simply a good bet not to live up to his draft position, even if his underwhelming production was in a top-tier league.
How Does This Apply to the 2022 Draft?
Now we can apply these findings as they relate to the 2022 NHL Entry Draft. There are two controversial prospects likely to be drafted high in the first round, Juraj Slafkovsky and Brad Lambert.
Slafkovsky and Lambert have really, really underwhelming point production. As a result, statistical models have been much lower on them than more traditional scouting.
The pushback against the statistical models is generally that it's perfectly fine that Slafkovsky and Lambert didn't produce very much because they played in a professional league and the fact that these guys played against such strong competition at such a young age is a good signal in and of itself.
As we have learned today, that is absolutely not true. Low-scoring players like Slafkovsky and Lambert are unlikely to live up to their draft position because of their production, and league quality does not make up for that.
We can be especially confident in this conclusion because if anything, my methodology would be biased in favor of more NHL-ready prospects because I am using seven year's post-draft rather than entire careers. Prospects from pro leagues should be more NHL-ready, so the lack of findings here is incredibly strong evidence against drafting the low-scoring pro league player.
There will be exceptions to every rule, but the general trend is clear: On its own league quality matters, but a great league is no excuse for bad production.
