Maximizing your testing

[freakish777]

Say you knew for a fact that at a given 60 person tournament, there would only be 1 Ichorid player.

Would this impact your deck choice for that tournament? Would it impact your sideboard choices?

Obviously it should. What I want to focus on is why and how it should impact your testing sessions.

Being that there's a limited amount of time, if you knew there were going to be a negligible amount of deck X at a tournament, I would argue that you shouldn't even bother testing the match up. The amount of time you spend testing a certain match up should be directly proportional to how prevalent that deck is in the metagame.

If Zoo represents close to 25% of the metagame and ANT represents close to 5% of the metagame, why would you spend just as much time testing against ANT when your expected return on investment (ROI) on that resource spent is 1/5th that of your expected ROI for the Zoo match?

At a 101 person, 7 round Tournament, if you're expecting 7 ANT players (I chose to go up to make math easier), your probability of being paired with one in round 1 is 7% (7/100). For the average deck (I'm not going to get into whether or not ANT posts average results or not), you can assume that good players will do well with it, average players will do average, and poor players will do poorly. At the start of Round 2, if 4 ANT players are 1-0, your likelihood of playing against one in the winners bracket is 8% (4/50). At the start of Round 3, if 2 ANT players are 2-0, your chances are 8% (2/25). At the start of Round 4, if 1 ANT players are 3-0, your chances are 7.69% (1/13).

This of course assumes you're winning each round, but the chances should remain largely the same, assuming the deck has a normal distribution.

So if all rounds have roughly a 7.66% chance of you running into ANT, that means 7.66% of your testing time should be devoted to that match up. If you've determined that your deck can't possibly beat ANT, and that you're willing to take the 7.66% risk of running into them, you shouldn't even bother with the match up, the same way you shouldn't bother wasting sideboard slots on the deck.


In general do you find your testing sessions mirror results, or do you find that your and your group just test every deck against every other deck?

[sunshine]

If Zoo represents close to 25% of the metagame and ANT represents close to 5% of the metagame, why would you spend just as much time testing against ANT when your expected return on investment (ROI) on that resource spent is 1/5th that of your expected ROI for the Zoo match?

This is somewhat misleading as it doesn't take into account the general effectiveness of your deck against AnT and Zoo respectively. Suppose your deck is a 70-30 favorite against Zoo and a 45-55 dog to AnT assuming average pilots. Would/Should this change your testing plan at all? Probably as the few percentage points you could gain against AnT are more likely to sway a match than those same few against Zoo. This doesn't mean that you should distribute your time strictly on expected MU percentages, just that they should be taken into account when trying to maximize the efficiency of your testing.

As for my own testing. I'll generally do most of my own testing against a gauntlet of expected decks almost indiscriminately - using a number of different decks. As a larger tourney approaches and I feel I have a good handle on what to expect my testing will be skewed to reflect that. I'm possibly not a great example though as I typically have long periods of time (~1 month) between larger tourneys so time is less of an issue.

[freakish777]

Would/Should this change your testing plan at all? Probably as the few percentage points you could gain against AnT are more likely to sway a match than those same few against Zoo.

I don't believe it should. If you have a 5% chances of playing against ANT at a tournament and a 25% chance of playing against Zoo at tournament, regardless of your match up percentages you should focus on the Zoo match ups.

Put another way, if your match up against Zoo is 70% in your favor when average players play both decks, and there's a 25% chance you'll play against Zoo in any given round, you want to give yourself as little chance of making play mistakes in the match you're going to play against most, know all the times you want to mulligan, when the roles change to where you're the beatdown instead of them, etc, etc. At the upper tables, you aren't going to be playing against average players. Furthermore, extensive testing in one match up, even if it's already good, will show you things you didn't know before.

Here's a good example, say that the deck right after Zoo is Reanimator at 20%, and your initial list, you estimated to be 60% in your favor. Now your test partners suddenly stumble upon a line of play that changes things (for instance, Exhume, you Crypt them and with Exhume still on the stack, they play Entomb), suddenly you realize your match up is now only 50% against them. This may lead you to re-thinking your SB. Alternatively, you may find that your match against Zoo is better than 70% with the amount of knowledge you have with the match up, and can afford to rearrange your main or sideboard to help with the other matches that aren't as good.


Of course rearranging your deck at that point, again, comes down to the percentage of the field playing that particular deck.


So hypothetically, if you have a 70% match against Zoo (25% of the field), a 50% match against Reanimator (20% of the field), a 50% match against Merfolk (15% of the field), a 30% chance against CBTop (10% of the field) and a 45% chance against ANT (5% of the field), and 50% against "random" decks (other 25% of the field), you have a 52.75% chance to win any given match up.

So you're slightly above average. Great. Just like you should configure your main deck and sideboard to ignore the matches you're less likely to face, you should spend less time on them as well. If the extra time you spend on the Zoo match up pushes it to 75% in your favor, and you lose out on 5% in the ANT favor because you know less about the match up, that takes you to 53.75% chance to win any given match up.