Optimizing for the wrong metric could hurt your bottom line in the long run, says Martin Gontovnikas, Co-Founder and GTM Advisor at HyperGrowth Partners, even if the optimization is successful. Sound counterintuitive? Gontovnikas has an example from his previous role as Senior Vice President of Marketing & Growth at Auth0.
“We were doing a lot of experiments on our activation emails, trying to optimize them for open and click rates, and we increased them a lot,” he says. “But we only realized a couple of months later that what we should have been optimizing for was activation. Once we looked further into that, we found that activation was actually worse for those who got the email versus those who didn’t get it.”
They realized then, says Gontovnikas, that every single one of their KPIs should be linked to the bottom line in some way, whether that’s in terms of revenue or customer retention, rather than testing for testing's sake. “In our case, the open and click rates weren’t related to revenue, but activation rates were. After that, we placed much more attention onto actually picking the right KPIs, because otherwise, you might spend months optimizing something which hurts your revenue.”
A new approach to experimentation and optimization
“Starting to think about how KPIs influence revenue will drive how you think about experiments and how you can improve them,” he says. Gontovnikas thinks most companies aren’t thinking about their KPIs through this lens, because, ultimately, they’re hesitant to set KPIs that might be harder to optimize for than metrics like website traffic and email engagement.
The easiest way to get started, says Gontovnikas, is to start every hypothesis by saying ‘We expect that doing X change will help us get to this X KPI in X timeframe’. “That format will help guide you on what metrics you need to be thinking about.”
But hypotheses don’t always need to be based on data, says Gontovnikas. It’s equally important for people to trust their instincts. “A lot of people are scared to go on their gut feeling because they’re worried they might be wrong and there’s a real sense of vulnerability. Because of that, they try to only use data. But when you’re starting a startup, you don’t have any quantitative data. And sometimes even if you do have quantitative data, you still don't have the exact events or metric you need.”
That’s where qualitative research comes in. “If you interview five or six people, your findings might not be statistically significant, but you get a feeling based on what people are saying what you think you should do,” he says. “But if you think that it's important, then you can build a really good hypothesis on that for your experiments.”
Why gut feelings don't always have to be based on experience
“I think it’s important for growth teams to have a mix of people who have a lot of experience and people who have no experience. The people who have experience will bring their knowledge of what they've done previously and what worked or didn't work, but they will also bring their biases on what can and cannot be done based on that experience.”
Those with no experience, on the other hand, won’t bring the same biases with them - which can be a huge competitive advantage for young start-ups. “They might come up with a crazy wild idea that it's actually fantastic and nobody has done it before,” says Gontovnikas. “In the past, something I’ve seen with a lot of hypotheses is people trying to do what everybody else is doing, but better. I have a mantra that it's better to do something different than it is to do it better.”
