Decks to Beat (2020-21) Discussion Thread

[H]

Yah that was more of a subjective take, but since it was from one of the better grinders I thought it was wor th mentioning.

If you are fine with the data collection project data you would just rank all the decks by the winrate CI min, which naturally takes into account sample size. Or if you believe tiers are popularity not power just rank it by play percentage. The time period could be anything you want.

Well, the current system was just a weighting of results. In other words, it was made to give one an idea what one would be most likely to see in a recent top 8. As such, it is heavily weighted (in fact, solely so) on top 8 results. Just going on win rate might give a similar result, but I am not exactly sure.

[Reeplcheep]

Delver’s performance in the super qualifier yesterday was the most dominant of any deck ever since the data tracking project began (I believe).

54 players (1/4 of the field), 215-164 record for a CI min of 53.6%

[H]

So, long time no update, which is totally my fault, but the old methodology didn't seem to really be working and I wasn't happy with the results I could get out of the limited techniques I had.

But the Legacy Date Collection project just put out an interesting compiled sheet for all of this year.

So, what does everyone think of the following "top 8" lists to be put into the DTB section as follows:


I think it "works" in the sense that those seem as if they'd be the top 8 decks you'd likely be seeing at top tables/in a given top 8.

Comments? Critques?

[Reeplcheep]

Why did you skip the ones not highlighted? Because it seems like by skipping the ones with bad winrates you are doing the same thing I suggested (sorting by CI min). I am on the data project and could even post the list myself for you every month if we are ok going by CI min.

I like CI min as an indicator since it naturally balances play rate and win rate to predict what you see in the top tables. A popular deck with a bad win rate will be eliminated due to its low mean win rate, and a fringe but successful deck will be eliminated due to the large confidence interval. In a tournament, popular bad decks should drop out as the matches go on and you are unlikely to hit fringe decks unless they are very very good.

[Reeplcheep]

As an example, the only difference between my method and yours would be that it would put Jeskai over Sneak & Show. S&S does have a higher win rate, but my method would argue that might be just because it has 3x less players and they may have gotten luckier. From a perspective of hitting it at the top tables, you are more likely to run into Jeskai. Even if sneak and show might be a bit better, 3x more people in the field will make a huge difference especially in the first 4-6 rounds.

[Reeplcheep]

Presumably if the point is “the decks to beat” you want information on what you need to beat to make top8. Imo, a top 8 based approach introduces more variation to the answer (fundamentally a smaller sample size). It also doesn’t answer the question very well: if Tony Scapone spikes 2 challenges do you really need ruby storm hate to top 8? If the field has tons of jeskai players going 50% you won’t see them in the top8 but you do need to beat them to get there.

[H]

I like CI min as an indicator since it naturally balances play rate and win rate to predict what you see in the top tables. A popular deck with a bad win rate will be eliminated due to its low mean win rate, and a fringe but successful deck will be eliminated due to the large confidence interval. In a tournament, popular bad decks should drop out as the matches go on and you are unlikely to hit fringe decks unless they are very very good.

Well, I am not statistician, so I just looked at that column first, because I know what it is saying.

What exactly is CI measuring and telling us? (Not a rhetorical question, I really don't know.) I am not averse to using it, I just want to know what it is saying.

[Reeplcheep]

Informally, The MWP is what the winrate is “most likely to actually be”.
The CI min basically tells you what is the “worst reasonably possible” the winrate could “actually be”.

If you go 7-2 with an unique brew the most likely winrate assuming nothing is 77%. But the sample size is small; you could just be lucky. It’s “reasonably” possible that the winrate is “actually” 45% and you got really lucky or it’s “actually” 94% and you got unlucky.

If elves pilots go 35-16 they most likely have a winrate of 69%, but it could “reasonably” be “actually” anywhere between 55% and 80%.

The CI min is the 45% and 55% numbers in the examples above. The precise definitions of “reasonably” and “actually” are where all the mess comes in, but the data project uses fairly rigorous ones.

[H]

Ah, ok, so CI is confidence interval. Makes sense now. I don't see why CI min would be worse to use, so I can do that.

[Reeplcheep]

Ok I will try to post a nice infographic soon.

[Reeplcheep]

Methodology is on my twitter.

[Phoenix Ignition]

Awesome data Reeplcheep! Although I don't really visit these forums enough or play legacy anymore, that's some really cool stuff.

[TonyRo]

It's encouraging that most of the top decks are fighting for a few percentage points. Thanks for your work!

[Reeplcheep]

There is a pretty vast gulf between the top 4 and the rest. 5% is a huge difference in a game with as much variance as magic.

[H]

So the new update should look like this, if we go with that data?

UR Delver
GW Depths (Turbo Depths thread is a bit far from this.)
Lands
Doomsday (Not finding a thread for this sort of thing.)
Death & Taxes
8Cast (This is the closest to a thread we have, I think.)
UWR Control (Oddly, we don't really have a thread for this sort of thing?)
4C Control

Well, this really highlights another issue, in that we don't really have threads for some of these archetypes, unfortunately.

[Reeplcheep]

Yes that looks right. The main difference is including jeskai over S&S and elves. It performs a bit worse but is the second most played deck. I think jeskai can just use miracles thread. It is UWx splashing for pyros and maybe a few EIs.

Doomsday, I will try to steal a primer from wonderpreaux.

GW maverick has often played depths as a flex slot, I think it is ok to use that thread.

[H]

I think jeskai can just use miracles thread. It is UWx splashing for pyros and maybe a few EIs.

It just doesn't seem quite right to me to have a thread called Miracles and full of historical lists that contain Miracle cards, but be for a archetype that now contains no Miracle cards.

I understand the genealogical progression, but still seems odd to me.

[TonyRo]

There is a pretty vast gulf between the top 4 and the rest. 5% is a huge difference in a game with as much variance as magic.

Not trying to be pedantic here, but I don't really know what you mean by this. If I created a game with higher variance and told you the best strategy yielded a 1% edge, would you find that more significant? EV is EV and doesn't really have much to do with variance unless you can't tolerate it, e.g. in a poker bankroll situation. Based on your data, the top 5 or so decks all roughly score within 1-2% of each other, which to me seems very reasonable, especially when people seem to really hate playing against Delver right now. Totally agree that picking something like TES over Delver or Doomsday does feel like a pretty seriously concession though, as the difference in scoring percentage is around an order of magnitude higher than other choices!

I say all that, but also would really like to see a ban or two and also see less Delver.

[Reeplcheep]

1. The above data is using very rigorous criteria, and we still found that delver was significantly better than the field. What more do you want?
2. 5% above the field is the usual criteria given for bans by WOTC.
3. Yes, I believe the less important player skill is, the more balanced the options need to be for metagame health. Magnus Carlsen will beat you with terrible openings like the Bongcloud Attack. But almost everyone is going to switch seats if a slots machine offers 0.5% better payout. Great players on great decks will win usually just 65% of their matches; and if the average player is gaining a third of that just by choosing delver it will cause metagame collapse. EV is EV but most people aren’t pure spikes. They will play other things if they feel it is reasonable for play skill to outwiegh the -EV.

[TonyRo]

1. The above data is using very rigorous criteria, and we still found that delver was significantly better than the field. What more do you want?
2. 5% above the field is the usual criteria given for bans by WOTC.
3. Yes, I believe the less important player skill is, the more balanced the options need to be for metagame health. Magnus Carlsen will beat you with terrible openings like the Bongcloud Attack. But almost everyone is going to switch seats if a slots machine offers 0.5% better payout. Great players on great decks will win usually just 65% of their matches; and if the average player is gaining a third of that just by choosing delver it will cause metagame collapse. EV is EV but most people aren’t pure spikes. They will play other things if they feel it is reasonable for play skill to outwiegh the -EV.

1. I don't want anything, I just have a different definition of "significantly" apparently. That's okay, no need to get testy.

2. Yes I know - as I said, I'm pro bans. I just find the scoring percentages to be less drastic than I would have thought. That was my only real point with my initial post. Thanks again to you and Joseph for doing the crunching.

3. Nice chess reference. Your point about playing other things is really at the heart of what I'm surprised by. With Legacy my feeling has always been that people love it so much because it feels like so much more is reasonably possible and out there to explore. Over any reasonable number of Magic games, I'd intuitively guess that you'd be hard pressed to play enough games to really "feel" the difference in scoring percentages with a lot of these decks, for instance on your list up through Stoneblade. You could play 100 matches over weeks and never notice. I wonder if a lot of the format frustration comes down to how games feel and intangibles that don't have anything to do with statistics and how much Delver actually wins. The interactivity and length of games, common play patterns, etc. But yeah coming from chess myself I'm used to really warped win percentages: if you're making Magnus Carlsen/Bongcloud references I'm sure you know that in chess even just 200 Elo means being something like a 75% favorite.

BTW, I think you're giving people that play slots far too much credit.