Adding monsters to the mythology

I don't want to be a bummer, but there really was something about first edition AD and D that was intuitive for the D.M.. When the creatures walked into a room, you sort of knew what kind of monsters just ought to be in there based on the creatures that populated your fantastic visions as spawned by movies, myths and books. So, you know about goblins, you get fairies and giants and crap like Rumpelstiltskin.

While I love the new Dungeons and Dragons(s, 3.0, 3.5, and 4.), where the F are they getting these monsters? Are they Mangan or something? Who, while planning a dungeon, thinks 'this would be a great place for some faceless mountain lions?' The end result is kind of like old Jorune or Call of Cthulhu where the monsters are crazy but you have no idea what they are and no frame of reference to remember them on the fly. Plus, with Jorune or Call of Cthulhu, you really only need a couple of monsters for the adventure. In Dungeons and Dragons, I'm seriously getting to the point where I just open the book to a random page and that's whats hanging out waiting for everybody. What's their motive? What does their lair look like that makes it unique to that creature?

Let's try it this way, old Dungeons and Dragons--the characters enter a forbiddon glen in the middle of the forest, slowly they creep forward until--
a--they hear the sound of pan flute and see a man with goat legs run by
b--the sound of hoofbeats shows a herd of creatures half man, half horse. They raise their bows because you have tread on their domain.
c--a nearby tree whips you and says, "who dares disturb my slumber."
d--a horrible spirit rises out of a forgotten grave and floats towards you angrilly.

New Dungeons and Dragons
a--a horrible giant with lightening for skin, says, "who dares my...what the hell am I doing in the middle of a glen with lightening for skin. What the hell, is this street fighter?"
b--you see a bunch of clockwork bugs examining a map of the continent trying to figure out which kingdom to take out. Luckilly you've caught them at step one of their master plan: conquer this forgotten glen with no discernible value.
c--you see a half eagle half horse creature that is perpetually on fire waiting for Spring to end so that it can start a forest fire...because....um....because... Anyway, you see it a mile off because it's on fire so you avoid the glen.
d--you hear the creeking of gears. My lord is that an iron golem?! Or a guardian...or maybe a hommunculi...or, who the hell keeps making these things...maybe its warforged...or...why are there so many robots in this game...and do they really eat old peoples' medicine? And, hey, how did this thing get in the middle of the woods.

When you know what the monsters are, you don't need to spend hours trying to figure out what should be in the next room and why. When you don't know, you have only two options--either the encounters are just crazy random (evil thing at the middle of the dungeon seems to attract everything to it and charm them so that they're all friendly to one another) or overprescribed (everything in this dungeon is a goblin). Personally, I think Dungeons and Dragons aught to either get back to using monsters from real legends or write up a hell of a lot more description. Obviously, some of the monsters like Sahuagin, have become classics by now, but some of the other creatures really need some description of where they might be found and what sorts of lairs they inhabit.

Strangely though, the real answer seems to be to custom build monsters with abandon. After all, if you know that you want some kind of fungal monstrosity, you're unlikely to find the right fungal monstrosity in the monster manuals--they'll probably have fire for skin or have some kind of connection to mind flayers. Your best bet, I think, is probably just to make the crap up. And why not...afr all, you don't have to worry about making a creature that is somehow out of character with the other monsters in the game which are, by now, so damn random that they don't make much sense anyways.