The different uses of KPIs and metrics – Making a better game versus raising money or securing a publisher

• Tweet
• Pin It

Many games companies struggle with how to use their metrics and Key Performance Indicators (KPIs) effectively.

They are conflicted because game metrics have two different purposes: Comparison and Improvement.

Improvement

I use metrics to help games companies make better games. My aim is not to compare one games company to another to see which one is better. For most entrepreneurs and businesses, this is not useful. The goal is for YOUR game or company to get better. Invidious comparisons with other games or businesses are not useful. And it is worth considering that you will rarely get the full picture about other businesses: they cherry-pick the data they share, so you get the “best picture”, not the real one. Too much comparison with other businesses or games is unhelpful.

My favourite startup blogger, Mark Suster, a successful entrepreneur and venture capitalist, has a phrase for this: Start-ups are all naked in the mirror. You know all of your own flaws, but you only see the best bits of your competitors or other games companies, who are revealing the best information about themselves, and are metaphorically all dressed up for a night on the time.

So your goal is to take your KPIs and make them better. Improve your own D1 retention or conversion rate or First Time User Experience (FTUE) rate. Don’t worry about your competitors. Be the best you can be.

A key advantage here is that it doesn’t matter whether you are using the same definitions as other people. Some people define retention as spot retention (of all the people who starting on Day 0, how many returned on exactly Day X) while others use Rolling Retention (of all the people who started on D0, what percentage returned on any day from DX onwards). If you measure differently to your competitors, the comparison is useless, but if you are only trying to make your own metrics better, as long as you don’t change the way YOU calculate the numbers, you can be confident in your results.

Comparison

On the other hand, investors and publishers DO compare you with other businesses. They are trying to make a decision about which games or companies will fit in their portfolio, so they analyse your Key Performance Indicators (KPIs) to make informed decisions about your current success and future trajectory.

They care about comparing your Retention Curve to the Holy Grail popularised by Supercall of 40/20/10, the Spot Retention numbers on D1/D7/D30 respectively. They care about your CPI (cost per install), conversion rate and ARPU (average revenue per user).

So if you are raising money or trying to secure money from a publisher, you may need to be clear about the definitions, and to compare yourself. But be aware that this has nothing to do with making a better game. Different genres have very different metrics, and even different games within a genre can be successful with a different blend of number of users, retention rates and patterns of spending behaviour.

Conclusion

The key message here is that you need to know WHY you are using metrics. If you are trying to make a better game or run a better business, focus inward. Use definitions of KPIs that are easy for you to access, to understand and to influence by changes you are making. Work to a better version of your current self.

If you are trying to raise money through investors or publishers, you need to understand the comparisons better. But remember that everyone (including you) will always cherry-pick the best data. Don’t be fooled into thinking everyone else is doing better than you. They, like you, are putting the best version of themselves out there.

Metrics are powerful tool for self-improvement. Don’t let them become a tool for invidious, inaccurate comparison with others.

Related Posts

• 

  China now generates higher ARPU from its games than the US, South Korea or the UK
  China now generates higher ARPU from its games than the US, South Korea or the UK
• 

  Benchmarks are bollocks?
  Benchmarks are bollocks?
• Gameforge revealed some stats – they are useful
  Gameforge revealed some stats – they are useful
• 

  Low conversion in free-to-play is a feature, not a bug
  Low conversion in free-to-play is a feature, not a bug