KPI Driven Development

What does KPI driven development mean at Traplight?

Here at Traplight we rely on data gathered from both the product and it’s marketing efforts to help us create high quality Free to Play (F2P) games. The Key Performance Indicator (KPI) driven development combines our game development experience with relevant, reliable data. Below we have opened our most common tests and KPIs that we use to validate our work, and to keep our products on the right track.

 

KPIs for Prototype and Engagement Betas

At the beginning of the project we test the game prototype with internal and external testers. We want to get initial feedback, check crash rates and get data about the first session length. On the marketing side we test our theme ideas with Click Through Rate (CTR) tests to find our potential core audience. The results will support the game team’s creative decisions when going forward with the project.

When the build first goes to store, even in one country, we have started what we call the Engagement Beta phase. The goal is to develop the game based on what data we need to get first, and then improve on those numbers with every subsequent update. The focus is first on day one and day three retention (D1, D3) and what we look more than just the separate numbers is the ratio between those two: The D3/D1 curve (and later D7/D3 or D28/D7) gives us a prediction of the longevity and appeal of our game.

In this phase we also focus on improving the session count and session lengths. The goal is to create an increasingly engaging and long living game with each update (up to 30 days of gameplay), without yet adding huge social features, live ops or monetisation. The basic game itself should keep players enthralled for 3-4 weeks at least.

But as said, our KPI approach is not limited to game development only. On marketing side we focus on finding the core audience for our game: we optimise for high CTR, and seek players with high engagement and retention. On the community the engagement numbers are also much more interesting than sheer volume at this point. We want to know how many players join our community through our in-game link, how actively are they chatting, how much feedback do we get and how many people react to our social media posts.

 

How we’ve done so far

With our current project Battle Legion we started out by building a core game with 5 units and no progression to test if it would be fun. We ran a small campaign with the goal of seeing daily user engagement and D1 retention. With each update we have added more content to the game, with the aim of getting a solid D1 first and getting better with each update. We’ve had a couple of places where the numbers went down, but because we only make a few changes at a time we are able to quickly asses what caused the drop, and get back on track in the next update. Below you can see a breakdown of some of our numbers so far on Android in Brazil.

 

Next steps: Monetisation and Scaling Up

We are moving on to monetisation tests soon using the same methods we have used so far. Instead of trying to build the whole monetisation and shop at once, we build it piece by piece and run tests in between. The goal is to build a monetisation that offers something for every player segment at every stage of the game, and to do this we need tests and data.

On Marketing and User Acquisition (UA) side we start to scale up and build Return of Investment (ROI) positive UA on multiple channels, while looking for the best performing creatives and audiences. Community and social media KPIs will focus on boosting monetisation through social events and community actions, and re-engaging segmented players that are on the cusp of churning.

The KPI driven development helps us lessen the weight of “what-if”s and “I-think”s during the development: We base our decisions on relevant, reliable data instead of hunches. Of course data and metrics alone don’t create an amazing game, but they guide to the right direction in our design process. The test results help us understand what the players truly want, and this is why having the right KPIs steering our development is so important to us.