Don't miss
  • 2,232
  • 6,844
  • 6097
  • 134
Joost Van Dreunen
Home   >   Analysis and Opinion   >   Analysis   >   Of Mice and Metrics

Of Mice and Metrics

By on April 19, 2011

SuperData Research is a research provider specialized in digital goods measurement. Using internal metrics analysis, KPIs and transaction-level data, it helps clients adopt, execute, and adjust their virtual item sales and micro-transaction revenue model; it believes that mastering your internal metrics is key to building a sustainable, profitable online entertainment company. In January 2010, SuperData Research secured multi-year seed funding. Its client base includes brand owners, developers, retailers, publishers, VCs, and payment providers. Joost Van Dreunen is its managing director.

Perhaps the most important question to ask before performing a data analysis of your game is “What do I want to know?” Before answering it, we should reflect on the paradox between the success of social game companies, which emphasize data-driven design, and the lack of enthusiasm this new breed has received from designers throughout the industry. The latest generation of game developers, the accusation goes, occupies itself much less with innovative game design. Instead, it focuses on acquiring as many new users as possible, retaining them for as long as they can, and, finally, monetizing this audience for all they’re worth.

Tiger Analytics

Today, the combined share of digital distribution, subscription and virtual item sales represent roughly a quarter (23%) of the total video game sales in the United States, up from 1% only ten years ago! The industry, in other words, is moving away from exclusively retail-based revenue to business models in which game companies have direct access to their end-users. That’s a big deal. Being on the creative, productive side of an entertainment business without having to face immediate, sometimes merciless feedback is an historic anomaly at best. Having to deal with this newfound direct relationship with end-users, presents at once an immediate challenge and opportunity for designers. In this digital environment, the rule of thumb is that only 40% of the actual work is completed at a game’s release.

Secondly, long before social game companies like Zynga and Playdom showed up, there existed data-driven success stories. EVE Online, a scifi-based downloadable MMO by CCP Games, for example, recruited its own economist. Together with his team of analysts, his job involves publishing a quarterly report on what’s going on in EVE Online’s economy. Similarly, virtual world IMVU employs a small army of data scientists. Their day job revolves around price elasticity, understanding seasonal variation, and a virtual item recommendation system that rivals that of online retailers. Analytics, in other words, are not some cheap innovation from recent years, but have been a key component in a sustainable game company for years.

Granted, it presents a steep learning curve. Generally, a company’s most talented programmers are assigned to the front-end of product development, leaving the job of watching the numbers on the back burner. It can be no surprise that dealing with an unattended analytics system is both daunting and annoying. And, in the absence of a disciplined approach involving a designated driver, development teams are almost literally inventing the wheel again and again.

Adonis Design

A growing number of game companies are starting to apply data-driven decisions in their overall business model. Experienced designers such as Brenda Brathwaite are currently making a case for using both game designers’ intuition and the quantified metrics provided by analytics toward a next generation of games. Distinguishing between the ‘why’ of player behavior and the ‘what’, she argues, analytics can be a helpful tool in approaching game design holistically. Other veteran designers that echo the same sentiment include Dan Daglow (Neverwinter Nights), Richard Garriot (Ultima) and John Romero (Doom).

Well, great. But what if I’m not a designer with decades of experience and funding? Then what do I do?

For one, it’s not rocket science. There’s no need to make things unnecessarily complicated. Building a simple funnel that shows you, for instance, what marketing channels are most effective is easy to set up and makes good business sense.

Second, it is important to build analytics in from the ground up. There is no reason why your 200-page design document shouldn’t include a few paragraphs on things you think you need to track. Start with the questions you want answered: “Where is my audience coming from?”, “What is the retention rate?”, etc. Adding a few tags after your game has gone live is setting yourself up for disappointment. Then, once data starts pouring in, regularly look at the output and see how it reflects on the design.

Third, make it someone’s job. The various success stories you’ll find throughout the industry generally have one thing in common: there’s a designated driver. This allows your analyst to learn what makes your audience tick, and make valuable recommendations to your design team.

Bi-Winning

The need to deal with data isn’t going anywhere. Whether we believe it is valuable and necessary component, or whether we believe it is just another thing that distracts us from doing our job, it is now part of our everyday. The reason, quite simply, is that data present us with one of those remarkable resources that, in its use, increases in volume.

Succinctly speaking, there are two types of metrics for each company. The first revolves around internal optimization. Specifically, this refers to building funnels and identifying relevant data points that allow a designer to do a better job. On this end, the always-working Albert Lai from Kontagent has been making a tremendous effort to show how companies can use metrics for optimization.

The second type of metrics needed in this new environment are so-called key performance indicators. In every other business people will ask, “How are we doing in relation to the overall market?” And game companies can be no exception. Before you embark on the development of a farm-based game for a social network, you probably want to investigate how crowded the overall space is, and what the margins are. And maybe it will even show you where that idea you had last year could break some ground today.

Despite initial resistance to the idea of analyzing data as part of overall game design, this will soon become a common practice throughout the industry. So next time you find yourself looking at a bunch of data wondering “What do I want to know?” remember that you want to know how to design better games.

About Joost Van Dreunen