Nicholas’s Guessing Game

• Tweet
• Pin It

Nicholas’s Guessing Game is a thought experiment that makes the design process faster and more efficient.

When you have identified a new feature that you want to build, ask team members to write down a guess for which metrics are likely to be impacted by this change and by how much. They don’t have to be KPIs; they could by diagnostic metrics such as daily logins or “amount of hard currency used in our game per day.” Guess whether the metric will go up or down, and by how much. “By 3%,” “by 5 percentage points” (i.e., from 10% to 15%) and “from 5 to 10” would all be valid guesses.

The guessing game forces people to build a mental model of the game and of the consequences of making the proposed change. They form a hypothesis—or guess—about how the change will affect a metric they can measure. They observe the experiment, a key component of the scientific process.

Ask each team member to explain their reasoning. It will give you insight into how the team thinks about the feature. It will teach you something about the feature and often about the entire game. It will also surface differences of opinion within the team because of different understandings of the nature or purpose of the feature, which you can address.

Write the metrics down on a white board and take an average. You will see the team’s perception of the consequences of a feature in this average. While working on Angry Birds Transformers, we were tasked with making a “Retention Update.” After a two-hour brainstorming session, we had a bunch of feature ideas ready to pass to the production team to determine costs and timings. I forced the group to play Nicholas’s Guessing Game, and every one of us—every one—guessed that all our ideas would drive up conversion and average spend but not retention. We kept all the ideas in the backlog for a future update and went back to the drawing board to work on retention.

Call it a Guessing Game, not forecasting, or estimating. (Feel free to call it [your name]’s Guessing Game, though.) If it is a forecast, people will refuse to play. They will say that they feel unqualified to participate or that the analytics team should forecast. This is a big mistake. The importance of the process is the formation of the mental model, the creation of a hypothesis and the discussion about the experiment. The value lies in the process, not the output. (Although the average from the Guessing Game is surprisingly often close to the experimental results.)

When the results come back, the people who guessed wrong won’t remember what they guessed. It is amazing how often they remember their guess when they were right.

This is an extract from Nicholas’s new book, The Pyramid of Game Design – get your copy here!