Think about it. What're we really looking for when we playtest stuff? Assuming the developer has half a brain, if he says its for 4th level characters, he won't throw in an encounter with Orcus, right? That's all you really need to have.
Everything else comes down to DM skills.
Game balance? Who cares? Not the OSR types. We just run the fuck away if its too tough, since the xp comes from gold and not kills anyhow. If something is totally out of whack, the DM adjusts on the fly anyhow, if he's any good. This goes back to the special snowflake thinking rampant in newer editions and story games. "Wahhhhhh... no fair, I died in the first encounter!"
Fuck you, you pussy. Back the fuck off from my game table.
Tricks and traps? So what? Either they'll figure it out and live, or they won't and they'll die. Big deal. Chances are they don't even have a name for their character until they're 5th level anyhow. Besides, playtesting that with one group just means that group was able to figure it out or not. As for other groups, their mileage may vary. If the trap is stupid, the DM should just skip it anyhow.
Reading the DM's room descriptions to players? Big whoop. We all wing that as DM's anyhow, regardless of what the boxed text says.
Game breaking exploit type shit? Hell, if the players figure out a way to make an end run around the encounter, or even the whole module, they get bonus xp from me if I'm DM'ing.
I guess the ultimate thing to playtest for is to answer the question "Is it fun?" Fun has little to do with the stats of monsters, game balance, room descriptions, exploitable stuff, or tricks and traps anyhow, and everything to do with the DM skills at improvising when all the shit hits the fan, as well as the chemistry between the DM and the group of players. That's completely subjective stuff.
So why do we playtest old school shit? For the word of mouth buzz and promotional marketing type gain I guess, and maybe to stroke the developer's ego. That's about the only thing I can think of. It seems unnecessary.
Castles & Crusades Week - So just about a year ago I asked for some advice on Castles & Crusades. The response was overwhelming and overwhelmingly positive. I went out and picked up ...
28 minutes ago