The Perils of Playtesting

In a couple of articles recently, I've argued that over-playtesting can lead to games which have a generic, bland "please everyone" feel about them. At one point at least I likened this process to film production. So I was very pleased to read the following in an article about crappy blockbuster films ...

The test-screening process - in which studios show their most expensive films to focus groups and then change them accordingly - also has to take some of the blame for the artistic failure of his movie, says the screenwriter. "They're going for the widest possible audience. So you end up having a movie that doesn't offend anyone and which everyone doesn't mind, instead of a movie some people love. But I never lost sight of the fact that I was happy to see it made."

Full article here:
www.guardian.co.uk/film/2009/jul/03/bloc...ael-bay-brett-ratner

The parallels should be obvious. Yes, ideally, in a gaming situation a designer, publisher or developer ought to have the strength to say "no"! But these people are only human - designers want to please gamers and publishers want as wide a market as possible. I remain unconvinced that extremely wide playtesting is a good idea. Again, the piece has another revelant quote:

"It's narcissism and power that ruin movies," agrees one veteran publicist. Over three decades, she has seen plenty of those ego clashes between producers, directors and stars. "A lot of producers really want to direct. And if the director is someone who's malleable, for whatever reason - maybe because he couldn't get the thing greenlit for 10 years - the power of the producer can corrupt."

Substituate "producer" for "publisher" and "director" for "designer".

It probably is difficult to draw to many parallels between different entertainment industries. But many movies that are made never see the theater and only play on DVD or on TV. Those are made for a broad audience, with no particularly high standards (but often a preference for genre). So, to cater to the lowest common denominator is not a bad business decision as such. It is definitely a business decision and that is what producers do. They are rarely into charity.

The movie business is much more grown up than the board gaming industry. While there are relatively many of private board game publishers, willing to work at sub-market returns, their number in movie makers is rather small (which is different from the situation that loads of people are willing to work at sub-market returns to be in the industry, which makes dime a dozen movies quite cheap to produce).

Also the boardgame publishers are by movie standards, quite small in budget. And for many designers, private publishing is a reasonable option (more so now than before the revolution in graphic design in the 1990s). This probably results in relatively stronger position for the designers vs the publishers.

I delay playtesting until I have a complete, finished game, and then foolishly expect that everybody will just love it and praise my genius. Instead, there will be a wide range of complaints and suggestions. Some complaints are very legitimate, pointing out specific problems that need to be fixed, and some suggestions are great, actually improving the game. Some are ridiculously petty or even useless, as they are focusing on elements of the prototype that will obviously be improved if the game is professionally published.

And then there are the suggestions that really frustrate me, the ones where somebody wants me to completely change my game into something else that doesn't even remotely interest me. It's even worse when that playtester insinuates that I am a bad game designer if I don't make those massive changes, or that my game will never get published unless I make those changes. It gets even worse if my design involves a licensed property, as I find myself getting dragged into fannish debates over minor characters X, Y and Z, and why they were included/excluded/misinterpreted.

So I've developed thick skin, and conversational deflection skills. I listen, but once the conversation wanders to far afield, I kind of stop listening and just make listening noises.

The tough part is that sometimes a publisher will ask for the same kind of massive changes, and then I really need to listen, and also very diplomatically choose my responses and revisions. And even then, some publishers will make additional changes in the final stretch between submission and publication, and that is when a designer can really lose control. Ultimately, if you have a very specific vision of the game you want to design, the only way to really see it through is to stick to your guns and then self-publish.

I playtested several CCGs back in the day, the lead playtester for Raw Deal was my primary opponent when that game was in it's heyday and we playtested L5R until just before they screwed people over with Gold Edition. I do a good bit of playtesting of board games now, both prototypes and soon-to-be-published stuff.

There are basically two approaches designers/developers take to playtesting. One is to ignore all the comments and the other is to try to please everyone, thus creating a bland game and sometimes destroying the good things the game had going for it in the first place. It makes playtesting the game pretty frustrating, no matter which route they take.