Last time, I talked about what a colossal pain in the ass it is for players when game mechanics don’t perform as advertised, and why that will cause your game to end up in the trash can. Today I’m going to talk a bit about how you can prevent that from happening. But, first thing’s first.
Game rules are not words from your heart.
For some reason, many game writers have this idea that when you write up some game rules, they are above criticism, without flaw, and inviolate. To some extent, that is true. True, in the same way it is true for trash novels that will never be published. The instant you decide that you want money for your game, you are simultaneously deciding that your game is a consumer product. Not art.
Secondly, people vastly overestimate the value of innovation and novelty in comparison to less sexy but much more important qualities like learning curve and consistency. People use words like “gimmicky” to describe games that have an interesting idea and nothing else. They use words like “fun” to describe games that are easy to pick up, easy to play, and deliver a consistent experience, regardless of how innovative they are from a rules standpoint. If you can’t make an innovative mechanic play as well as the less exciting alternatives, broom it and move on.
On to the practical matters. There are a lot of ways to test a game out, but there are some tried-and-true stages you’ll probably want to go through.
Do your homework first.
You can do a lot of testing just by gaming out various scenarios under the rules and seeing what happens. If you’re writing an RPG, get four guys (or however many are the recommended) and start coming up with some scenarios and acceptance criteria. For example, you might have some test encounters where the expectation is that for all of them, the players win after at least four but no more than seven combat turns and at least thirty but no more than sixty minutes. Then, come up with some “test parties” of benchmark characters and start running encounters. Run the same encounter two or three times, then switch the characters and go again. If you’re writing a wargame, come up with some sample army lists and start playing scenarios. One trick is to have the players switch sides (for a wargame), player positions (for a card or board game), or characters (for an RPG) after each turn and see what they think of the situation from the other side of the table. Make sure to include the GM in any switching procedure… they’re a player too, despite how infrequently people remember that.
Another key point is that, for this or any other kind of formal testing, you need a third party (not the GM) to referee the test, take notes, and switch the test scenarios when things get bogged down. His job is also to make sure the participants remember that this is a test, not a regular play session. Part of that means not getting attached to what is happening on the table, and part of it means being willing to rip holes in your own work and that of the other people sitting there with you. Internal testing should be the most critical of the material, out of any of the phases, because external testers will often be reluctant to point out flaws in your work while you’re standing over their shoulders.
The purpose of this kind of testing is two fold: first, to iron out the basic mechanics, find inconsistencies, and surface submerged defects. Is the game taking too long to play? Is one type of character or unit totally ineffective? Is a rule too hard to follow, or does it allow for multiple interpretations? Are people having trouble keeping track of all the options that are available to them? And second, to start generating the test scripts and scenarios you’ll use when you test with people from the outside.
I strongly recommend omitting any kind of “discretionary” mechanics or conflict resolution systems (for rules disputes) that you are intending to include in the final product. Mechanics like action points and stunt dice can be great, but they’re basically just a lever players or GMs can pull when the game isn’t doing what they want it to do on its own. You should test these out later, but they’re just going to obscure underlying problems at this stage.
Also, make sure you thoroughly test the set-up tasks, character advancement, any kind of tournament scenarios or game-to-game continuity rules, and other grab-bag mechanics in the same way. It’s pretty easy to lose sight of those rules that aren’t used as much when you get into the meat of making sure the core mechanics are functional, but they need the same kind of attention.
Get fresh eyes on the material.
After you’re fairly confident that the bulk of the rules make sense, play at the speed you want, and don’t contain any glaring flaws, it’s time to get people outside the core team to test the game. You can do this at a con, your local store, whatever. You might be surprised how easy it is to get people to sit down and test for a while… it’s actually harder to make people understand that you are testing and not just demonstrating. Again, have someone present to moderate the test just like in the first step.
This is the stage where you want to find out if the rules are accessible, not just internally consistent and coherent. This is the best time to find issues in character generation, army list building, board-game set-up phases, and so on. It is critically important that you get actual beginners to do blind testing of this stuff, because you and anyone else directly involved in the game will fly right through it without seeing the problems.
Similarly, your in-house group will develop a common understanding of how the rules work very quickly. You may find, however, that another group of people will read the same instructions and get a very different idea of how the game is played. If the testers start arguing with each other, that is a pretty solid sign that your rules haven’t been written clearly enough. This is just as true for Dungeons and Dragons as it is for Apples to Apples or Scattergories.
For adversarial games, thoroughly test the end-game. Set up a game in progress scenario and have some inexperienced people play it out to completion. One of the most frustrating things about adversarial games is when the outcome has basically been decided but the game shows no signs of ending. Make sure that even first time players can seal the deal in a reasonable amount of time.
If you’re not familiar with usability testing protocols, an easy way to get more information out of players at this stage is to tell them to think out loud. When they are going through character generation or army listing (which absolutely should be tested independently of the table-play itself), have them just say whatever they’re thinking. Prompt them if they stop talking. If they get completely stuck (more than one minute without progress), move them to the next stage on the script. And, by the way, have a script. You don’t need to follow it exactly, but you need to be able to skip forward if the testers can’t advance on their own. If you’re testing character creation and there are three steps (say, attributes, skills, and powers), have sheets on hand that have attributes finished, attributes and skills finished, and are completely finished with attributes, skills, and powers written up. If something breaks down and the testers can’t advance, move on to the next check point and hand them the pre-generated sheets.
And finally, a piece of advice: you’re testing the game, not soliciting suggestions. Don’t ask people what they think is wrong with the game. They don’t know, any more than you do. Have them play it, watch, and you’ll find out where the problems are. Incidentally, this is the bright line between a test session and a focus group. You can and should do both if you can manage it, but make sure you have the proper compartmentalization so that focus grouping doesn’t bleed into your test session.
Side note: Keep track of defects
Any time a tester has a problem, immediately write it down with a brief summary. Later, put it in a centralized spreadsheet and keep track of whether it’s been fixed or if you have decided not to address it. You don’t need to change something to respond to every complaint, but it is a good idea to at least know what’s been discovered and have a single location to track it all. Some issues are easy to fix, and you need to address them. Spelling errors, rules phrasing that people interpret differently, and other “broken” content should be your highest priority. If players seem to having trouble selecting their characters’ skills, or deploying their army, or showing some other common area of frustration, make sure you have a plan to address it.
This is important no matter how small your operation is, but it’s a make or break issue if someone besides the original test moderator is going to be responsible for fixing the problems reported.
Get lots more eyes on the material.
This is the part where you would want to distribute play-test materials to external testers who won’t be directly supervised. The reason you want to do this is pretty simple: once people get experienced with the game, they are going to start finding tricks and end-runs in the rules, some of which might ruin the game. If you want words like “replay value” or “competitive scene” to apply to your game, you need to squash these problems with extreme prejudice.
However, you will probably find very quickly that this type of external play-testing is less than completely successful at finding the sort of problems I’m talking about. Unless you have a massive play-test program, you’re just not going to be able to get enough people to try out the material to find everything.
At this point you might say, “Hey asshole, wasn’t the entire purpose of this post to tell me how to find that stuff?”
The fact is, your game is going to go out with problems in it. The first edition of anything does. Eventually, people are going to find stuff in the rules that you never did. The only question is whether that will happen before they get deeply invested in it, or after. If it’s the former, your game wasn’t tested properly and will end up in the customers’ trash cans and on the tomb-shelves of distributors, where shitty games go to die. If it’s the latter, you’re on the right track.
Next time I’m going to talk about how to design with revision in mind, and some of the techniques and technologies you might consider exploiting so that you can fix your game after it’s gone out the door.
