Get a Feedback Loop and Listen to It

In an article on playtesting,¹ Matthew Burns points out how unpredictable humans are:

Nobody really disputes that playtesting in some form or another is indispensable in order to make good games, of course. Understanding how an audience may react is tricky even in a linear medium such as the novel. At least there, we can assume readers will start at page one and continue to the end. In games, though, our agency enables our habits, and out habits become our blinders. You don’t consciously know that you always strafe to the right to avoid grenades– you just do, and your design ends up reflecting that. The choice then is to ship it that way, or to show the game to a lot of people, some of whom will instinctively dodge to the left or backpeadal or who might try to bunny-hop over the damage radius (something you’ve never thought to do).

On the concern that too much testing interferes with the designer's ability to be creative, he writes:

These are legitimate concerns. But it is also important to remember that as personal as they can be, art, entertainment and video games are all transactions: things that occur, somehow, in the space between the creator and the audience. The playtest is a tool, one that has evolved to help us grapple with that single most important quality of games, the reason they are so beguiling and why they are so problematic: their interactivity.

Used properly, playtesting does not tell you what to do, so much as it tells you what you have in front of you. It shines a light into the possibility space of the game– a light of a different color or from a different angle than you are used to, one that makes possible a better understanding of its true shape. In this sense, a game designer’s artistry is not thwarted by the playtest. The artistry is present in how he or she reacts to the results. Data is just data, and it is up to us to decide what we will do about it.

In related news, via Rémy Rakic, a great blog about game design.