In addition to the problems that make non-combo decking inviable, there are already tons of insanely powerful answers to it. Milling with some sort of Brain Freeze combo usually lets you win on the spot and is generally capable of generating enough mana and draw to find answers to their problem cards. Imagine if Jotun Grunt read like this:

1W
Cumulative Upkeep -- You gain 2 life.
4/4

I mean really, just a few turns could completely ruin your day, never mind the fact that it can beat you to death.

Gaea's Blessing is a pain too, anyone can board a singleton of it and you'll have to spend huge amounts of resources to deal with it, like inefficient mill such as Extract, or stuff like Stifle that neither mills nor helps against Goyfs/Stalkers/etc. If you hit a Blessing, it's like all of your gameplay up until that point just got undone. The only need to board one to have a good chance of it buying them enough time to kill you, and if you make an actual good mill deck, people WILL board that one Blessing.

There's plenty of other random stuff that'll ruin your day too. Anyone playing Academy Ruins can run a Feldon's Cane, and you'll help them "tutor" it up. I imagine that Ichorid is an auto-loss.