How Not to Design Your Game System, Part 4: Killing Trees Won’t Bring Back Your God Damned Money

Today I am going to write a few words about why publishing your homebrew RPG book is a bad idea. Well, that’s not fair. It’s a terrible idea. For one thing, it’s a book.

A book has to be written around the limitations of putting paper in a binding. Namely:

• The pages need to be in a fixed order, so the content needs to be structured around that fixed order. You can’t easily create multiple task-flows that use the same content in different contexts. If you expand the content, you’re stuck with the structure of your first publication.
• There’s no capability to seamlessly cross reference information from one part of the book to another. Side bars, boxes, callouts, and references to go look at another page are all band-aids put over the top of the basic problem.
• You can’t change the content after publication, short of publishing a new edition. Errata documents are what you get when people who are tied up in the process of print publication realize that you can push an updated version of your content to the web in about five seconds, but don’t follow that capability to its ultimate ends.
• You need to worry about all the irritating vagaries of print production. Pages breaking incorrectly, tables running out of bounds, graphics not rendering correctly, and so on are all artifacts of the print production process. Don’t use print production software unless you are planning to put a book in hard copy. And for the love of God, never source your original content out of your print production software.
• You can’t design the rules with interactive enhancements in mind. Several companies are releasing software toolkits to go along with their rulebooks, but as far as I know, no one has gone the final step and unified the two areas. It is not difficult to put all the rules for character creation into the same context as the character creation utility you are providing, but big companies won’t do it since they are used to selling the rulebook and then selling the tools. That’s great when you can keep the money you used to get just for the book and then pile on a subscription fee for the tools as well, but small publishers don’t have that luxury. And, when you step away from the need to produce a printed copy, other capabilities become possible.
• Producing a print book, which is the only reason to write in the book format to begin with, is a capital investment. Even Print on Demand set-ups are going to jack up the price of your product and raise the barrier to entry. You’re not benefiting from print unless you’re already an established publishing company. Hint: you’re not.

If you’re trying to produce a book (or a PDF, which is a file format for those aspiring to produce a book), you’re probably using some software tools like Word (or OpenOffice) and Acrobat, or an XPS writer. The first thing you should understand is that a tool like Word is a WYSIWYG editor for a markup language.  If you don’t understand what’s going on inside your tools on at least a rudimentary level, you can’t really do anything with your source files other than produce crippled electronic imitations of print books. So, let’s take a field trip over to the magical land of the Microsoft language specification for Office Open XML! Fortunately, ISO makes the standard available in four easy ZIP files.

Part 1: http://standards.iso.org/ittf/PubliclyAvailableStandards/c059575_ISO_IEC_29500-1_2011.zip

Part 2: http://standards.iso.org/ittf/PubliclyAvailableStandards/c059576_ISO_IEC_29500-2_2011.zip

Part 3: http://standards.iso.org/ittf/PubliclyAvailableStandards/c059577_ISO_IEC_29500-3_2011.zip

Part 4: http://standards.iso.org/ittf/PubliclyAvailableStandards/c059578_ISO_IEC_29500-4_2011.zip

Part 1 is a mere 5,588 pages! And you’ll be happy to know that Microsoft has considerately refused to completely adopt their own specification, so Word does not fully comply with these standards.

At this point, you are probably wondering what all this garbage has to do with producing and distributing small-scale RPG products. The problem is this: the tools you’re probably using are so hypercapable that most people have no idea what they can really do with them, and so difficult to understand that most people give up long before they can leverage the real capabilities they have available. The bigger problem is that these formats are only accessible through heavy-weight, opaque authoring tools like Word that don’t give you an easy transition from WYSIWYG “dumb editing” to a real understanding of the format. And you need a real understanding of the format you author in if you want to do something cool with your content, like turn a flat page of text describing character creation into a fully functional character generator and advancement tool. Or turn a page of text about war-game combat into an online turn tracker that tells you what moves and in what order. Or automatically optimizes your character’s ork murdering abilities with the weapons you specify in a handy drop-down menu.

So throw your tools away. We live in an age of freely available authoring tools (and supporting tool-chains) for light-weight, easily comprehensible languages like DocBook, DITA, and good old fashioned HTML. You can set yourself up to author in one of these formats in a couple of hours (for free) and transform your source to attractive output in a few more with style sheets. Web content management systems are a dime a dozen. Instead of setting up a cargo cult around the print distribution system, play to your strengths: you have no legacy customers to placate, no legacy content to manage, and no legacy business agreements to honor. No one in your shop is going to start a turf war with you when they realize digital publishing obsoletes their job. If you don’t feel liberated by that, you should.

So why do you want a distribution middle-man if you’re going to do all your business online? Payment processing? PayPal and other options are far from perfect, but they’re a hell of a lot better than a distributor that takes a big cut of your earnings and gives you no direct access to your own customers. Publicity? I’m sure the authors of all of the thousands of RPG PDFs on sites like DriveThru RPG are cashing fat checks thanks to all the publicity they get. Experience? Well, they’re pretty good at extracting enough revenue from the corpses of a thousand failed dreams to keep their own doors open, so they’ve got that going for them.

More to the point, distribution chains assume a fixed, boxed product like a PDF or ebook. For all the reasons I already talked about, that’s a really counterproductive thing for a small outfit to produce. An actual print book isn’t quite as bad (at least you are hamstringing yourself for a reason), but they’re far more cost-intensive. Either way, putting out your content as a book essentially puts you in the same game as Wizards of the Coast or (insert the big player in your genre here) and you’re never going to be better than they are at producing a book. You can think of alternatives as apps, or services, or site memberships, but they’re all different methods to the same end: getting out of the cage that print publication puts you in. You can take yourself out of that box entirely.

All you need to do is let go of the idea that producing a game means writing a book.

How Not to Design a Game System, Part 3: Polishing Your Turd

Last time, I talked about what a colossal pain in the ass it is for players when game mechanics don’t perform as advertised, and why that will cause your game to end up in the trash can. Today I’m going to talk a bit about how you can prevent that from happening. But, first thing’s first.

Game rules are not words from your heart.

For some reason, many game writers have this idea that when you write up some game rules, they are above criticism, without flaw, and inviolate. To some extent, that is true. True, in the same way it is true for trash novels that will never be published. The instant you decide that you want money for your game, you are simultaneously deciding that your game is a consumer product. Not art.

Secondly, people vastly overestimate the value of innovation and novelty in comparison to less sexy but much more important qualities like learning curve and consistency. People use words like “gimmicky” to describe games that have an interesting idea and nothing else. They use words like “fun” to describe games that are easy to pick up, easy to play, and deliver a consistent experience, regardless of how innovative they are from a rules standpoint. If you can’t make an innovative mechanic play as well as the less exciting alternatives, broom it and move on.

On to the practical matters. There are a lot of ways to test a game out, but there are some tried-and-true stages you’ll probably want to go through.

Do your homework first.

You can do a lot of testing just by gaming out various scenarios under the rules and seeing what happens. If you’re writing an RPG, get four guys (or however many are the recommended) and start coming up with some scenarios and acceptance criteria. For example, you might have some test encounters where the expectation is that for all of them, the players win after at least four but no more than seven combat turns and at least thirty but no more than sixty minutes. Then, come up with some “test parties” of benchmark characters and start running encounters. Run the same encounter two or three times, then switch the characters and go again. If you’re writing a wargame, come up with some sample army lists and start playing scenarios. One trick is to have the players switch sides (for a wargame), player positions (for a card or board game), or characters (for an RPG) after each turn and see what they think of the situation from the other side of the table. Make sure to include the GM in any switching procedure… they’re a player too, despite how infrequently people remember that.

Another key point is that, for this or any other kind of formal testing, you need a third party (not the GM) to referee the test, take notes, and switch the test scenarios when things get bogged down. His  job is also to make sure the participants remember that this is a test, not a regular play session. Part of that means not getting attached to what is happening on the table, and part of it means being willing to rip holes in your own work and that of the other people sitting there with you. Internal testing should be the most critical of the material, out of any of the phases, because external testers will often be reluctant to point out flaws in your work while you’re standing over their shoulders.

The purpose of this kind of testing is two fold: first, to iron out the basic mechanics, find inconsistencies, and surface submerged defects. Is the game taking too long to play? Is one type of character or unit totally ineffective? Is a rule too hard to follow, or does it allow for multiple interpretations? Are people having trouble keeping track of all the options that are available to them? And second, to start generating the test scripts and scenarios you’ll use when you test with people from the outside.

I strongly recommend omitting any kind of “discretionary” mechanics or conflict resolution systems (for rules disputes) that you are intending to include in the final product. Mechanics like action points and stunt dice can be great, but they’re basically just a lever players or GMs can pull when the game isn’t doing what they want it to do on its own. You should test these out later, but they’re just going to obscure underlying problems at this stage.

Also, make sure you thoroughly test the set-up tasks, character advancement, any kind of tournament scenarios or game-to-game continuity rules, and other grab-bag mechanics in the same way. It’s pretty easy to lose sight of those rules that aren’t used as much when you get into the meat of making sure the core mechanics are functional, but they need the same kind of attention.

Get fresh eyes on the material.

After you’re fairly confident that the bulk of the rules make sense, play at the speed you want, and don’t contain any glaring flaws, it’s time to get people outside the core team to test the game. You can do this at a con, your local store, whatever. You might be surprised how easy it is to get people to sit down and test for a while… it’s actually harder to make people understand that you are testing and not just demonstrating. Again, have someone present to moderate the test just like in the first step.

This is the stage where you want to find out if the rules are accessible,  not just internally consistent and coherent. This is the best time to find issues in character generation, army list building, board-game set-up phases, and so on. It is critically important that you get actual beginners to do blind testing of this stuff, because you and anyone else directly involved in the game will fly right through it without seeing the problems.

Similarly, your in-house group will develop a common understanding of how the rules work very quickly. You may find, however, that another group of people will read the same instructions and get a very different idea of how the game is played. If the testers start arguing with each other, that is a pretty solid sign that your rules haven’t been written clearly enough. This is just as true for Dungeons and Dragons as it is for Apples to Apples or Scattergories.

For adversarial games, thoroughly test the end-game. Set up a game in progress scenario and have some inexperienced people play it out to completion. One of the most frustrating things about adversarial games is when the outcome has basically been decided but the game shows no signs of ending. Make sure that even first time players can seal the deal in a reasonable amount of time.

If you’re not familiar with usability testing protocols, an easy way to get more information out of players at this stage is to tell them to think out loud. When they are going through character generation or army listing (which absolutely should be tested independently of the table-play itself), have them just say whatever they’re thinking. Prompt them if they stop talking. If they get completely stuck (more than one minute without progress), move them to the next stage on the script. And, by the way, have a script. You don’t need to follow it exactly, but you need to be able to skip forward if the testers can’t advance on their own. If you’re testing character creation and there are three steps (say, attributes, skills, and powers), have sheets on hand that have attributes finished, attributes and skills finished, and are completely finished with attributes, skills, and powers written up. If something breaks down and the testers can’t advance, move on to the next check point and hand them the pre-generated sheets.

And finally, a piece of advice: you’re testing the game, not soliciting suggestions. Don’t ask people what they think is wrong with the game. They don’t know, any more than you do. Have them play it, watch, and you’ll find out where the problems are. Incidentally, this is the bright line between a test session and a focus group. You can and should do both if you can manage it, but make sure you have the proper compartmentalization so that focus grouping doesn’t bleed into your test session.

Side note: Keep track of defects

Any time a tester has a problem, immediately write it down with a brief summary. Later, put it in a centralized spreadsheet and keep track of whether it’s been fixed or if you have decided not to address it. You don’t need to change something to respond to every complaint, but it is a good idea to at least know what’s been discovered and have a single location to track it all. Some issues are easy to fix, and you need to address them. Spelling errors, rules phrasing that people interpret differently, and other “broken” content should be your highest priority. If players seem to having trouble selecting their characters’ skills, or deploying their army, or showing some other common area of frustration, make sure you have a plan to address it.

This is important no matter how small your operation is, but it’s a make or break issue if someone besides the original test moderator is going to be responsible for fixing the problems reported.

Get lots more eyes on the material.

This is the part where you would want to distribute play-test materials to external testers who won’t be directly supervised. The reason you want to do this is pretty simple: once people get experienced with the game, they are going to start finding tricks and end-runs in the rules, some of which might ruin the game.  If you want words like “replay value” or “competitive scene” to apply to your game, you need to squash these problems with extreme prejudice.

However, you will probably find very quickly that this type of external play-testing is less than completely successful at finding the sort of problems I’m talking about. Unless you have a massive play-test program, you’re just not going to be able to get enough people to try out the material to find everything.

At this point you might say, “Hey asshole, wasn’t the entire purpose of this post to tell me how to find that stuff?”

The fact is, your game is going to go out with problems in it. The first edition of anything does. Eventually, people are going to find stuff in the rules that you never did. The only question is whether that will happen before they get deeply invested in it, or after. If it’s the former, your game wasn’t tested properly and will end up in the customers’ trash cans and on the tomb-shelves of distributors, where shitty games go to die. If it’s the latter, you’re on the right track.

Next time I’m going to talk about how to design with revision in mind, and some of the techniques and technologies you might consider exploiting so that you can fix your game after it’s gone out the door.

How Not to Design a Game System, Part 2: Consider Three Mile Island

Today, I’m going to write a bit about what I see as the biggest obstacle a game has to adoption, after it overcomes the hurdle of getting into a player’s hands in the first place. In a nutshell, the problem is that of expectations not matching results. Consider the following:

This is a control panel for a nuclear power plant. It is very large, very complicated, and allows a user to execute literally thousands of potential actions and millions of action chains.

Nuclear reactor design is instructive because the designers of very complex industrial systems (known in other circles as User Experience, Usability, User Interface Design, or Human Factors) have understood for a long time that unexpected emergent behavior can be a critical problem. If I flip three switches, expecting the result to be a green light that indicates everything is swell, and then the building explodes, the result did not match my expectations. So let’s consider Three Mile Island.

The cause of the Three Mile Island disaster was a little light on a control panel that indicated whether an emergency relief valve was open or closed. That’s what the operators had been trained to think, anyway. In the event of a pressure emergency, the valve was supposed to open briefly, then close automatically, and that’s exactly what the control panel showed had happened.

Unfortunately, the light was not hooked up to a sensor that was actually measuring whether the valve was open or closed- instead, it was measuring whether the solenoid that operated the valve was receiving electricity or not. Because of a mechanical fault, the valve became stuck open even when the solenoid was shut down and the light on the control panel was dark. So the operators saw that a problem had happened, saw the light come on and then go dark again, and continued on under the impression that everything had worked correctly.

The problem was not the design of the reactor, or the design of the control panel, or the sum complexity of the system, it was simply this: the designers did not account for a mode where the valve was open (due to mechanical fault) even when the operating solenoid was turned off (or alternatively) the operations training did not communicate to the operators that the control panel light was ambiguous in that mode.

Now consider this:

Where am I going with this? Well, Dungeons and Dragons is in roughly the same class, complexity-wise, as a nuclear reactor. There are thousands of possible actions and millions of action chains that a user can perform on the system, and the system produces a result. If the results of some action or actions fail to match the player’s expectations about the outcome, they will simply go do something else that isn’t so frustrating.  The underlying issue is that the player does something with the belief that they will get a specific result, and when it doesn’t happen they will re-evaluate whether the thing they are trying to achieve is worth the cognitive effort required for further analysis and another attempt. When the reward is supposed to be a fun gaming experience (as opposed to a salary, or a school grade, or some other tangible), the frustration threshold is staggeringly low. If you’ve ever heard a non-gamer friend describe something as “stupid” or ” a waste of time” or “boring,” chances are that they’ve simply run out of desire to understand the machine you are trying to get them to play with.

And it’s hard to blame them. Who would want to play a game after making a character who is a “Fighter” with “Power Attack” only to find out that they are less capable of killing enemies than the healer? Or, for the GURPS fan, after discovering that their “Weapon Master” is substantially less skilled with a weapon than someone who didn’t pick up the advantage, and that they only get a minor damage increase in exchange? Obviously, a more complex system is more susceptible to this problem than a less complex system. But even relatively simple board games can trap people by dashing their entirely reasonable expectations with seemingly nonsense results. Consider Settlers of Catan, where one player making a non-optimal decision at the set-up phase can turn the entire game into a drawn-out slog because of resource starvation.

One of the worst offenses a game can commit in this vein is example characters, strategies, or army lists that do not work well real play. Some poor sucker reading Shadowrun 4th Edition’s examples might not even realize that the most critical trait a character can possess to increase their combat capability is extra initiative passes (getting to go more times in a combat round). Someone reading GURPS might not realize just how critical it is to have the advantage High Pain Threshold if you want to take a hit in combat and live. Someone playing Warmachine back in First Edition might have been misled into thinking that all those giant steam powered robots on the cover would be useful in a tournament.

Good job spending 30 bucks on that miniature, idiot

Privateer Press had the wherewithal to tear it all down and build again. Few companies do.

These things happen because the designers didn’t really understand their own creations. It’s not that they’re bad designers, necessarily. It’s that emergent behavior is something that is emergent. You can’t fully predict every consequence of every rule or mechanic without a staggering amount of simulation. Some of the most intellectually taxing games, like Chess or Go, are such because they are just simple enough that a raw computational solution is just barely viable (for human beings) as a way to predict the course of the game. Consider the difficulty of doing that with an RPG or tabletop wargame. It shouldn’t be surprising that games have emergent behavior that isn’t predicted by the designers. Just like that misleading indicator light at Three Mile Island, the game has modes that aren’t fully understood before they start causing people grief.

The challenge for a game designer is that, unlike a nuclear reactor control panel, the common (and appropriate) reaction from a new player to a game where results do not match expectations is to drop that game in the trash and find another one. Or watch TV. Next time I’ll talk about the the only practical way to avoid this sad state of affairs: testing.

What is the Model Train Ghetto?

Today guest poster LogicalPhallacy joins us to explain exactly what is meant by the “Model Train Ghetto,” and share the most hilarious image of John Lithgow I have ever seen.

So if you’ve been reading this site, you’ve most certainly seen the term “Model Train Ghetto” tossed around, or references to model trains. I think it’s high time that we discussed that. You see model trains are a definitive example of an insular hobby. To get into model trains requires an investment. A real model railway can easily total in the hundreds of dollars, and of course there are multiple competing standards of rail gauge, train size, props etc.. All this together means that model trains are a bitch to get into.

This is a shame because what was once a hobby that would bring joy to a child’s eyes on Christmas morning, is now an industry that caters to a rather select few.  Even the Wikipedia article is a mountain of technical terms and details about the different competing standards. Where the model train community might describe itself as refined, exclusive, and well known, most would describe it as expensive, lonely, and the butt of many jokes.

RPGs are running a risk of becoming model trains. How so?

Historically there was a time when model trains were a popular gift, the sort of thing that you give a child to occupy their time, or to play with with their friends. Then over time the trains got more complex, and the hobby got more and more focused. The kids grew up and kept their model trains, but not too many new kids came in.  Now we see an interesting situation: it is hard to find a model train set for a kid to just get into, and you don’t go looking in a toy store for one, you go to a hobby shop.

Likewise, there was a time when an RPG (“Red Box” Dungeons and Dragons) was a popular gift, the sort of thing that you give a child to occupy the time, or to play with friends. Then over time the games got more varied, and the hobby got more and more focused. The kids grew up and kept their games, but not as many new kids came in. I’ll stop there to avoid belaboring the point, but the parallels are pretty clear.

So what are RPGs to do? Well that is where this blog comes in. There are some people who actively want RPGs to be like model trains. They frequent forums to talk about the good old days, and how to relive them best and most accurately. They actively shun newcomers and argue for a more “exclusive” game. In this hobby we tend to call them grognards, but we might as well call them conductors.

How not to design a game system, Part 1: Drag me to hell

Today we welcome guest columnist TheOldestMan!

Bad business decisions happen every single day, in every industry, everywhere. Most small businesses fail because the people running them have little business sense and either can’t commit or can’t pull together the necessary capital to keep their project running long enough to get into the black. This principle is no different for game companies than it is for any other kind of company. That’s not my area of expertise, and I’m not here to talk about that. I’m here to talk about the actual product, the game, and what makes some games good and others bad.