FanPost

Moderating Lottery Outcomes, an Entry Draft Proposal

Dear NHL, at last week's GM Meeting, you invited the submission of proposals to revamp the Entry Draft Lottery. Here's mine...

Why is it worth reconsidering the current draft lottery structure? Through the lottery, the worst teams have a high probability of suffering a modest fall in draft position while the better teams have a low probability of reaping a pretty big reward. On an expected basis, the weak teams are subsidizing a big payoff for one (or two) lucky strong teams. This can be seen in the following graphs.

Struggling teams are set up to fall in the lottery. The worst seven teams have a greater than 50% of falling. Those orange bars below towards the left are just too large:

dbff1a8543e45e287e916b38d4b4acda.0.png

As shown above, despite the chances being low for any individual team to climb in the draft, when they do, the jump is massive. Undeserving teams are poised for big gains if they are selected to move up. Those blue bars below towards the right are just too large:

2d9ccfc25495f8b09c4303e684c98735.0.png

When viewed on an expected basis (multiplying the two graphs), the picture painted is too steep a price being paid by the weak teams to the strong teams.

633cf549581d20a20108fb02efe43757.0.png

Summing the "area under the line" in the above chart, the five worst teams in the league are expected to collectively concede 5 slots in the draft to the ten best teams in the lottery. The biggest loser is the worst team in the league, bearing the brunt of the transfer cost with an expected loss in draft position of 2 slots. The best half of the lottery teams are each expected to gain over half a slot. A more equitable system would significantly flatten this line. The proposal described herein gets to that more desirable result. (As Seattle joins the league after next season, all discussion of the lottery from here forward assumes 16 teams/picks.)

848a84e8994a603c4da48df7667d4d89.0.png

To arrive at this proposal, the following guiding principles were used:

1. No team should fall more than 3 slots (same as current)

2. Teams should be limited to gains in draft position of 3 slots with minor exception --€” see last principle

3. The most likely outcome for a team should be not to move draft position

4. Where possible, a team should have a roughly 33% chance of gaining position

5. Where possible, a team should have a roughly 33% chance of losing position

6. Given a team is moving up or down, the average of slots moved should be approximately 1.5 in either direction

7. The worst team should have no more than a 40% chance at the #1 pick

8. Worse teams should have greater probabilities for higher picks than better teams (same as current)

9. The progression of probabilities across "the grid" summarizing the distribution should be smooth

10. As a consequence of principles 7-9 (and in contradiction to principle 2), the worst eight teams should have a chance at the top 4 picks

And, how did these principles come to achieve the ‘flatter' expected change in position curve? The principles are demonstrated in the below probability grid produced by the random process underlying the proposal:

3969a8cddb61b081f4444e9e3ae289d1.0.png

The diagonal of the grid is the highest value in each row and column (i.e, the mode of distribution for each individual team or pick). This means the most likely outcome is for a team to neither climb nor fall. The diagonal value varies but is on average roughly 35%. On either side, the distribution is reasonably symmetric giving a roughly 30-37% of no gain or loss in position. The probabilities spread 3 slots in each direction away from the diagonal, except for the expansion to the left of the fifth, sixth, seventh, and eighth worst teams up to the top pick. (Note that this exception is made with only the lowest of probabilities). The probabilities are smooth and monotonic away from the diagonal (i.e, no dips or jagged changes).

These takeaways can be seen more starkly by overlaying the diagonal peaks on top of each other to align for change from pre-lottery position. The below graphs do that first for the eight worst teams and then for the eight best team (I'm breaking this in half to avoid too much colored spaghetti on a single graph). Each line is a row in the grid, representing an individual team, with the diagonal peak value "centered" at zero.

a137c7c14f252783aacab71973fdcba4.0.pngabdcd2f66ec6401f69d5d2303393e036.0.png

The above graphic demonstrates equity in two ways. For a given team, there is relative symmetry in the profile of moving up or down. Across teams, the distributions are comparable with quite similar profiles.

Another way to see this healthy amount of equity in results is the relative flatness of the following graph. Shown below is the probability of moving up or down 2 or more draft slots (the bars corresponding to the vertical axis on the left) as well as the remaining probability of being within 1 draft slot of pre-lottery position (the line corresponding to the vertical axis on the right). For the most part, teams are neither overly exposed to extreme moves nor saddled with downside outweighing upside (or vice versa), but obviously this cannot be the case for the very worst or best teams as their distributions are truncated by the top and bottom of the lottery.

5da6add424a4e03fa751d06220ef6243.0.png

The following heat map shows the difference in probabilities of the proposal relative to the current lottery (i.e., the delta in grids).

ff442de067e656791ee13c0959e223cd.0.png

The increases from the current lottery (green) are for small gains from the diagonal where small upside in draft position is now more likely. The decreases from the current lottery (orange and, to some extent, yellows) are for mitigation of the big downside for the worst teams and elimination of the big gains no longer available to the best teams. For example:

  • the worst team's probability of falling to pick #4 has decreased by 40 points with roughly half of that being shifted to picks #1 and #2 (increased by 22 and 15 points, respectively) -- see first row
  • the ninth worst team's upside into the picks #1, #2, #3 is eliminated while its probability of moving up one slot to #8 has increased by 21 points from zero under the current format -- see ninth row

Focusing further on the change from the current lottery, across all teams:

  • the probability of losing position decreased 10 points, from 44% to 34%
  • the probability of gaining position increased 18 points from 15% to 33%
  • average slots moved up, when moving up, decreased 5 from 6.6 to 1.6
  • average slots moved down, when moving down, was virtually unchanged at 1.5

The proposal shifts FROM the current lottery's high frequency of moderate downside coupled with low frequency of high upside TO moderate frequency of both moderate upside and downside.

So, what is the underlying random process in this proposal? Unfortunately, the benefits of increased equitable treatment come with complexity as further described below.

This lottery proposal is actually the result of three lotteries. Lottery ‘alpha' is a scaled down version of the current lottery with the top 4 picks (#1-4) being allocated to the worst eight teams only. Lottery ‘beta' is an independent lottery (again a scaled down version of the current) where the bottom 4 picks (#13-16) are allocated to the best seven teams. Lottery ‘gamma' is dependent on the outcome of lotteries ‘alpha' and ‘beta' where the middle 8 picks (#5-12) are allocated to those teams not selected in the other lotteries. Furthermore, lottery ‘gamma' is more accurately described as eight mini-random processes that are dependent upon each other to shuffle and rank order those remaining eight teams into their draft position subject to the max 3-slot up-or-down move principle.

Because each of lotteries ‘alpha' and ‘beta' are independent, the probabilities of teams earning picks #1-4 and #13-16 are determined a priori (i.e., they are determined by direct input). However, due to the dependence on the ‘alpha' and ‘beta' outcomes as well as the interdependencies within lottery ‘gamma', probabilities of teams earning picks #5-12 can only be known ex post simulation. These ex post probabilities are determined by inputs to the lottery ‘gamma' process but are only an indirect result of those inputs.

Further detail on each of the ‘alpha', ‘beta', and ‘gamma' lotteries is as follows:

  • Alpha -- the top 4 picks allocated to the eight worst teams
    • 4 lottery balls (0-9) with the 10,000 draws results mapped to outcomes in proportion to the joint probabilities of a given 4-pick permutation
    • probabilities determined by input
    • 840 possible permutations but only 510 relevant for a 1-in-10,000 draw
    • scaling preformed to force the output from the relevant permutations to the theoretical input, within tolerance
  • Beta -- the bottom 4 picks allocated to the best seven teams
    • 4 lottery balls (0-9) with the 10,000 possible draws mapped to outcomes in proportion to the joint probabilities of a given 4-pick permutation
    • probabilities determined by input
    • 175 possible permutations but only 149 relevant for a 1-in-10,000 draw
    • scaling preformed to force the output from the relevant permutations to the theoretical input, within tolerance
  • Gamma -- the middle 8 picks allocated to those teams not selected in either Alpha or Beta (note this will always include the ninth worst team)
    • teams preliminarily assigned A, B, C, D, E, F, G, and H in pre-lottery order with A corresponding to pick #5 and H to pick #12 (the "pre-gamma position")
    • 8 lottery balls (0-9), each drawn to determine the raw movement of each team up or down from the pre-gamma position
    • for a given draw, the 10 possible results mapped by input to tenths of probability for the seven outcomes of movement from that team's respective pre-gamma position: 3, 2, 1 slot up; no change; or, 1, 2, 3 slot down (specifically, 0.2 for each of the three central outcomes and 0.1 for each of the four more extreme outcomes)
    • each team's A-H pre-gamma position is adjusted with the respective team's random slot move outcome to determine a preliminary position (the "post-gamma position")
    • the post gamma positions are ranked in order with any tie being broken in favor of the worser team (the "rank ordered post-gamma position")
    • the rank ordered post-gamma positions are subject to the 3-slot maximum move from pre-lottery position (the "bounded post-gamma position")
    • the bounded post-gamma positions are ranked in order with any tie being broken in favor of the worser team (the "re-ranked bounded post-gamma position")
    • the prior two steps (bounding and re-ranking with tie break) are repeated as necessary until there is no change in position

The grid of probabilities for this proposal was produced by simulating the lottery 30,000 times. As expected, the results for picks #1-4 and #13-16 (the four leftmost and rightmost columns in the grid, respectively) converged to the a priori probabilities of the alpha and beta input. The ex post probabilities for picks #5-12 (the middle eight columns) were an indirect result of all other inputs.

Having designed the proposal and performed the necessary simulation to test its results, a few observations can be made:

  • Assumed Goals of the Lottery are still achieved
    • A Disincentive to Tanking? The lottery should be one tool that the league uses to discourage uncompetitive play by a team to improve its draft position. This proposal achieves this but certainly takes a step towards reducing that disincentive. Nevertheless, the variability to which the worst teams are exposed should be a contributing factor against aiming to finish last. In other words, no position is guaranteed, and no position at the top of the draft has a greater than 40% outcome.
    • Entertainment Value? This proposal continues to give each team in the process the hope of moving up through the spectacle of a random outcome. (Having watched the Rangers finish out of playoff contention these past three years, I can say I looked forward to the draft lottery --€” with some very satisfying recent results!) Variability across sixteen teams is maintained, but the big prize of a top pick is certainly now limited to a smaller subset comprised of the more needy teams.
  • Tweaking -- It is not too difficult to reduce or increase probabilities under the proposal, particularly at the left and right of the grid (in the alpha and beta lotteries). However, any increase requires a decrease, and the tweaking process is really about shifting outcomes around the grid. Focusing at the left (alpha lottery), principles clash with another quite quickly for any large changes. For example, if the top left 40% is reduced too much, (a) the most likely outcome for the worst team (its mode) will shift down to pick #2, (b) adjusting upward the probabilities of other teams for pick #1 cause cascading effects up against the diagonal when the smoothness principle is upheld OR adding additional teams into the alpha lottery (even with very low probabilities) further contradicts the bounded 3-slot principle.
  • Alure of the 'Shuffle' Effect -- Lottery gamma presents the attractive idea of randomly moving upward or downward from a pre-lottery position. It sounds simple but is difficult to implement when the principle of limiting the move comes into play. Furthermore, the interdependence means it is challenging to achieve a desired probability distribution outcome both because of the indirect influence of the inputs as well as the cascading effects that can only observed through simulation. This proposal attempts to limit these difficulties to the middle of the lottery because the grid can be more easily adjusted through the alpha/beta process at the top and bottom (where some would say it matters most, particularly the top). If lottery gamma could have been further restricted (i.e., applied to less than eight teams) without introducing more theoretical outcomes to the alpha/beta lotteries (and therefore complexity), this path would have been pursued.

So, NHL, what do you think? Any questions? Feel free to come find me -- if you contact the masthead at blueshirtbanter.com, I'm sure Joe or Mike can send word to me. Meanwhile, I'll be lurking here at 'the Banter'...

(All underlying probabilities for the current draft structure have been sourced from tankathon.com.)