How a Data Jedi Measures and Analyzes Metrics to Power UA & Growth with Tenjin’s Matthaus Krzykowski

Aug 27, 2017

From the paid marketing metrics you need to watch and measure in order to bid more aggressively, to the new ways you merge data sets to power UA and growth. These are some of the key questions our host and Chief Content Officer Peggy Anne Salz explores with Matthaüs Krzykowski, Director of Platform at Tenjin and all-around Data Jedi.


MOBILE GROWTH: Today ,our guest is Matthaus Krzykowski. He is the Director of Platform at Tenjin. We’ll go over more about his career in a bit. But for now, to describe him, he’s an all-around “Data Jedi” with a long track record in data-driven companies and activities, as well as a frequent speaker at Mobile Growth events. So great to have you here, Matthaus!

MATTHÄUS: Hi Peggy, good to talk to you again.

MOBILE GROWTH: Our paths do indeed cross. They cross again and again, because it’s really all about data, which is of course your area of focus. Let’s first start with a top-level view of Tenjin.

MATTHÄUS: Exactly. Data is the key word. What Tenjin does is provide growth data infrastructure that helps people acquire profitable end users for their app.

MOBILE GROWTH: Let’s talk about acquiring users for your app. We’re halfway into 2017. A lot has changed. How your clients are approaching growth has probably also changed. I’d like to hear about the shift here. What are they focusing on now, compared to where they saw the opportunity, the low hanging fruit, at the beginning of the year?

MATTHÄUS: I think there a couple of new channels coming in like Apple Search, which people were experimental about, and also Snapchat. But I think the biggest thing that happened is competition in some of these channels. A lot of the UA managers thought, “I will just do more of the same”. But it turned out – especially on some of the networks which aren’t giving as much data as they used to so – the competition is very different. So what does that mean? What does a UA manager do on a Monday? If it’s an account ad network, I write an email to my account manager and say, “Hey, give me 2500 installs at $1.50 CPI”. Then a week later I get a result. Right? I think this what people were comfortable with a year ago. But now people have very specific strategies, such as asking, “Dear account manager, please bid $1.50 for VS, or $1.20 for UK on IS, or similar measures on Android”. Behind this strategy, there’s a lot of thinking on how much they could be bidding in a channel to get as much revenue or whatever their goals are for the app. This is, I think, what has shifted – the game has become way more “docu-centric”, and the UA manager must bid much more specifically to get to the success they need.

MOBILE GROWTH: That’s clear. It’s really about bidding more aggressively and having a greater grasp of the data. Now that’s in your backyard. Being at Tenjin you’re focused on the data, but what are you seeing? What’s your view of the paid marketing metrics to watch and measure in order to bid more aggressively and successfully?

MATTHÄUS: I think last year probably was a lot about LTV and mid size developers getting to the LT gain and having some basic LTV calculations. I think this has become way more of a thing. If I have a run and hit game, then I do a 3 day LTV prediction. If a user comes in based only on freebased behavior, I must come to the conclusion, whoever this is a campaign for, if I want to increase my budgets or not. Or 7 days, if it’s more, or you have the evergreen apps and games, where people have a half year LTV. I think LTV prediction has become way more popular. What’s really changed is that a year ago people took LTV from third party vendors, but now people understand way more what the app or game is about. So if it’s a racing game, it will have a very specific type of LTV. If your racing games have longer LTVs, such as 180 days, or if they have a very specific way to monetize, then it’s much more IAP and less ETRA than you. I think people have come up with better ways as to how they calculate this, because then they will be able to come up with the strategy that I mentioned before: $1.50 for yes, $1.20 for UK on the yes. This way, they are way more confident on statements like this, and can bid way more aggressively for that.

MOBILE GROWTH: That’s a great point. We’re not just focused on LTV, mother of all metrics. We’re focused on the curve of LTV. Then with subscription apps coming on strong, it’ll be very different for them. That’s really a focus. How are you helping your clients merge data sets that will power UA growth, given that we’re more focused on LTV?

MATTHÄUS: People have been doing this in our industry for a couple of years. Usually to build something like this costs a lot. The pieces are attribution, aggregation, in all the different channels, and analytics, which is user level data. In the past, people have been trying to combine these three pieces on their own. What we do is have a product and give people access to it in one go. We make it very easy and accessible to people, so they can do their tricks, which they must have in order to compete. So you can do this without us, but for us we’re cheaper and easier to get.

MOBILE GROWTH: So you talk about tricks, and one of these is also about the approach to the data. I know from listening to you, that you’re a big on lookalikes, cohorts, and getting that part of the data, extracting data from that part of the data, which people are just getting their heads around. At a high level, where do app managers get started?

MATTHÄUS: App managers don’t need to write their own, but they must learn and get better to manage data scientists. A simple step to start is to do some behavior analysis of users. What does a valuable user do? You can look at the first 24 hours, because you need to identify this. You may find, for example, that a paying user has more than 10 sessions in the first 24 hours, while a non-paying one has only 5 sessions. Insight like this can make you understand that maybe you should try to things like localize for more than 10 sessions and experiment around that. A deep, thorough understanding of your users separates people in the local like and Facebook game.

MOBILE GROWTH: So at a high level, understand the key characteristics of your most valuable users, and once you’ve understood those key characteristics, shape a UA strategy around that. If you’re finding your most valuable users are doing 5 sessions in two days or whatever it might be, then use that metric to guide you. Is that correct?

MATTHÄUS: Exactly. Especially, not only Facebook. You have other ad networks which provide user level data to people. But essentially, if you don’t know these things you’ll be guessing with your campaigns. For instance, Facebook, please give me users which make me more money, more like $2. But everyone competes in your genre. If you start competing differently you’ll suddenly have very different results.

MOBILE GROWTH: So continuing that thread, what is your advice? We know how you’re supposed to do it with some tactics here. How should I be really embracing this?

MATTHÄUS: Sure, I think a beautiful phase from one of our clients who has 2 of the top 20 grossing apps, ‘my user level data is really my treasure’. How I combine user level data with revenue data and how I am able, for example, to use this data for more titles in this genre, this is what my company is based on. This is what we’ve seen; if you’re a developer and your first game is pretty good, let’s say $10 million in revenue, it’s great. But if you really want to build a company, then you should have this data and a thorough understanding of this data. Because once you launch a second game, let’s say simulation game or farming game, you won’t be starting from scratch, you will be able to build on your success and be a couple of steps further towards building a company.

MOBILE GROWTH: Great segway way to success – let’s stay on that. So if you only had $10 to spend (of course you’ve got more than that as an app manager), where and how would you spend it to grow?

MATTHÄUS: I think probably on a lookalike, a lookalike based on some behavior analysis like the one I suggested. Then I’d probably run two $5 spend tests on seeing if the session example works, or something else that ties to unique data pieces from my end, such as the number of friends the user might have had.

MOBILE GROWTH: It’s interesting, you just don’t spit out a network, because when you ask it of app developers, app companies. The response you get is, “oh yeah, Apple Search or I’ll do it on Facebook”. But no, you have a very data-driven approach. I guess that’s a sign that going where everybody else is going is not a great way to achieve sustainable mobile growth.

MATTHÄUS: Exactly. I think we see more and more ad networks which give this Facebook type of level data to their publishers. People would be surprised there’s probably more than 10 by now. They’re just not usually well known. I think the ad networks feel the pressure from Facebook and Google. I see the trend to them giving more data access to people.

MOBILE GROWTH: Let’s stay with that. Finally, I want to wrap up with the elephant in the room. You’re talking about ad networks and I’m reading a lot about mobile fraud, in fact we all are. We’re hearing about which networks named are actually only fraud. So there’s a lot going on out there. What are your tips around what to watch in the data, or what to do about the data. Because ultimately, it’s about how to work with the data to recognize fraud and to measure it. What’s your take on that?

MATTHÄUS: While I think the noise around that is deserved, what it really is about is giving people a democratic access to tactics. That’s how to combat mobile fraud. The big guys actually don’t have a problem with this at all. Once they have access to your growth data you can do weekly or monthly tests. When you see irregularities, you send an email to your ad networks saying, ‘look guys, this doesn’t add up.’ You push back. We have people who on a weekly basis have a review of the data. Then they go back to the ad networks and they say, “sorry this is a mistake”, and they put the ad networks on the spot, including Facebook. They get huge rebates. Sometimes if you spend more than a million, it’s $100-$200,000 that people get back from the networks once they catch it. Again, if you have access to your growth data, I don’t think you need most of these tools, because you can tell the ad networks if something is not adding up. These tools are making it more accessible to people to combat mobile fraud with interfaces.