Carl Tsukahara, CMO @ Optimizely; Rob Pegoraro, Journalist @ Yahoo
Ascent Conference 2019
Rob Pegoraro [00:02:38] Good afternoon, everybody, I’m Rob Pegoraro, I’m a freelance journalist for Yahoo finance USA Today, where we got her fast company, your name here, and Carl one. You introduce yourself.
Carl Tsukahara [00:02:49] Sure. My name is Carl Tsukahara. I’m the CMO at OPTIMIZELY. If you don’t know what we do, we are experience optimization company that really helps both Challenger brands and established companies make sure that every deployment of an experienced and digital actually works. And so we use experimentation and personalization technologies to do that for about 2000 companies.
Rob Pegoraro [00:03:12] I’d like to know we have optimized the timing of this panel. This is before launch, not after. So none of you will be in the throes of a food coma. So. So, yeah. Let’s talk about what your company does. Give me a case study or two that sort of shows the difference you’ve been able to make.
Carl Tsukahara [00:03:26] Sure. So obviously we work with some large brands. So the IBM State Farm’s piece of the world, but also a lot of, you know, Challenger brands we work with people like Stitch, Fix, Kessler’s mattress, Atlassian, Salesforce.com, Artecoll, Salesforce.com, a Challenger brand, I think at this point. But so.
Rob Pegoraro [00:03:44] We have the tallest building in San Francisco.
Carl Tsukahara [00:03:46] Yeah. So if you think about The Wall Street Journal, since they’re local, you know, there have been a customer of ours for some time. Obviously, all the media companies have to go through a transition. Right. You know this obviously through your background. Well, going through from a physical to digital environment. And so as they kind of made this transition to want to acquire and retain digital subscribers, they thought about the whole digital journey and how they could make every touchpoint, whether it’s awareness, building, pre funnel, paywall, checkout. How do you make each one of those touch points, either from a front end perspective, visually colors, layout, language, or even in back in things where you’re looking at things like package composition recommendations as they’re going through this funnel. But this really helped them a lot. They improved their conversion of digital subscriber acquisition by about 60 percent. So we do that for a lot of different folks, but across channels. So it could be web in their case, it’s across Web mobile. We have a lot of media companies doing this. And if you’re trying to drive stickiness, if you’re a media company like CBS, driving stickiness in your environment for working for cord cutting.
Rob Pegoraro [00:04:51] So the whole newspaper paywall thing is fascinating to me because so many sites are terrible at it. I was reading a story from a paper in Virginia earlier this morning, got about a third of the way through. There was the pop up to subscribe to their newsletter. Got another third of the way through. There was the pop to subscribe where, you know, maybe you want to let me read to the end. So I’m curious, how do you decide in that case between, you know, get the short term win of getting someone’s name on a mailing list or subscribing and then the possible reputational damage of the site is a pain to read. And I’ll just read some crappy rewriting of it on a blog somewhere.
Carl Tsukahara [00:05:30] That’s a great question. I mean, I think, you know, not to kind of have a commercial here, but test it. And one of the principles of experimentation or testing is you can try different variations of an entire experience flow for different types of users or consumers. And it’s kind of a low risk way to look at things like conversion versus sea sat or, you know, potential churn risk. So you can look at things like how well are people converting versus their bounce rate? And you can put different types of the experience through, you know, sometimes really low volumes of your traffic five percent, but create enough kind of mathematical significance. You kind of know if you roll this out of scale, you’re going to get this kind of return on investment against different KPIs or behaviors you want. And so, you know, this is why I think the personalization testing is so important together, because we’ve seen a lot of brands roll things out without thinking about testing, without thinking about even doing a variation test versus what’s in place today. And sometimes it could be really damaging, not just your brand, but actually to your business. I mean, we have a large airline that had they rolled out a new part of their website experience, I would have lost about three percent of their bookings revenue from ecommerce. That’s a huge number when you’re a multibillion dollar entity. So test user data get the data. Obviously, you know, direct user testing is also really important. So get qualitative feedback. But at the end of the day, before you deploy, it’s really good, in our opinion, to create a testing culture. And so you’re always thinking about agility testing data and just making sure that you’re avoiding risk. But whatever experience changes are going to roll out in this case in the Journal, this both in the acquisition flow, but even post acquisition, they know there’s a bunch of actions that a subscriber will take that really drives long term retention habits of their subscriber base. And so there’s a number of things you can do if when you really think about us as all consumers, think about all the ways we interface with the brand, you know, in their digital flow. With that, that’s just one example.
Rob Pegoraro [00:07:32] And I guess the flip side of that, once you have someone as a customer, you want to personalize the experience. You don’t want to get too creepy or too clingy and you know it, what’s the KPI for that? How do you know you’ve stayed on the right side of that line?
Carl Tsukahara [00:07:46] Yeah, I mean, this is always the debate right now, right, about privacy and personalization. But generally, our brands tell us and, you know, I think we’re fortunate that we have a lot of, you know, pretty large and small consumer facing and also B2B brands. They basically tell us that because of, you know, what marketers do, we’re blasting people with email and retargeting and things like that. As long as you’re relevant, as long as you’re really relevant, then personalization can really work at scale. And, you know, just one example is you work with Crate and Barrel. And so they’ve been doing a lot of personalization things on their site, but they’ve been really thinking about this with the focus on intent, you know, because there’s so many ways you can personalize. But if you’re really kind of keeping ahead of your customers and what they’re trying to do and not necessarily using all historical bucketing to figure out what they did do, then I think you can really provide a highly relevant experience to them. And again, and then if you test it before you roll it out to every single person that fits a persona, you’ll know if it’s going to do what you want it to do. And so that’s why we think about these two pieces together. But we’ve seen the effectiveness of personalization. I mean, I think Crate and Barrel lowered their bounce rate by 20 or 30 points on their site. So there’s a lot of great things that you can do. But we always encourage this kind of combo platter of testing, thinking about intent, making it highly relevant, but then making sure you’re testing all the way.
Rob Pegoraro [00:09:04] So I have to ask, we’re sort of in the market for a new sofa. How many times do we need to look at sofas on Crate and Barrel dotcom before they send me a 20 percent off coupon?
Carl Tsukahara [00:09:12] Yeah. So all those things, you know, matter. You tell me the sofa story. That was pretty interesting because we do help them with things like that, like personalization in different category areas. So, yeah, no, I think there’s there’s all of those variants. So that’s the key thing to think about. You can test all those variants. And, you know, there’s always a point where the annoyance factor might get too high and you get that noise. But, you know, people either in your e-commerce business or your marketing piece or even for that matter, your developers or your product folks that actually own the coding and deployment of the final experience all of those folks can test and you can test even really small things. We’ve seen things as small as button colors change, a landing page conversion rate by 20 or 30 points. There’s all kinds of things you can test is create small variations, you know, like even in the SOFA conversion thing that that we talked about earlier. Like, it’s it’s a really great opportunity to really find out what’s going to work if you can scale up. And that’s probably the biggest key. If you test one thing a month, you don’t get much value. But if you start to really think about is kind of a micro discipline in your business, whether it’s in marketing or on the product side, you can see we’ve seen, you know, one use case at the at the Journal I mentioned earlier, they were able to create some actions around retention actions that helped drive conversion up by seven x seven percent. So you can get a lot of great benefits, both large chunks, but also incrementally to to get to the outcome you want.
Rob Pegoraro [00:10:36] I have to ask what what is a good or bad color for? Is mob just completely off limits?
Carl Tsukahara [00:10:42] Yeah. No black buttons on a black background. That’s a good attempt.
Rob Pegoraro [00:10:45] Good. So in our email back and forth planning this, we talked a little bit about what aspects of the customer lifecycle are at least optimized today. And I guess on the other hand, we talk for lifecycle versus the one time customer experience.
Carl Tsukahara [00:11:04] Yeah, so, I mean, I think the way to think of this, the answer, the short answer I think to this is that we you know, again, we have some really large companies with giant digital teams, giant product teams thinking about, you know, multi-channel experience. So but if you think of us as consumers, you flip it around that way. We touch a brand through a website, both the front and the back end. And by that I mean the front end is visual elements. The back end could be things like pricing or promotion packages. Right. We touch websites. We touch mobile apps. We touch mobile web. And we’re even doing things with chat bots now. And so I think there are very few customers or brands that have really gone through this entire optimization experience. And because it’s hard, it’s hard to predict what consumers will do. But again, there’s so many ways and you know, and then just think about the use case I talked about earlier, which is kind of pre acquisition. You know, maybe once you get a little bit cookie, you’re trying to drive them through a checkout flow. You’re trying to drive share of wallet or recurring sales. There’s so many different variations of how you watch your consumers to interact with your digital presence that, you know, we find that most companies, with some exceptions, like we work with Sky TV, they’re pretty mature. Others are less so. But most people, I think, are still, I would say not getting started, but in the very early stages of really using test and learn to know. And again, this is the whole point is to take the guesswork out and know what will work so that when your teams deploy, it actually drives a predictable outcome.
Rob Pegoraro [00:12:36] Since you mentioned Cookie, I have to ask, that’s become really unpopular now at Apple and I guess the Mozilla Firefox developers as well. Has that gotten in the way? I mean, it seems like here you’re trying to help the customer be happier in their transactional experience. This is not about following me around the web. Yeah.
Carl Tsukahara [00:12:55] Now, this is not about retargeting. This is really about what happens when they land with you. And again, that could come from a lot of different sources that could come from organic, that could come from paid, it could come from a pre or post authenticated experience, or in some cases you don’t know that much about them, you know. So even if you’re using reverse IP lookup to basically trying to figure out generally what IP they’re coming from. But the point is, is that you can if you think about those, those are all kind of micro segments of your business. And so no matter how you want to look at privacy and, you know, we do business with a lot of customers in Europe where GDP is superstrong. But no matter how you look at that, you know, if you align this discipline, whether it’s personalization or experimentation with the key things you’re trying to do as a business, I mean, that’s what it kind of boils down to, to making this program successful, consistent with the theme of what we’re discussing today. And I’ll give you an example. Like we work with the really large car rental company. And, you know, they’ve got a goal to add this many new car renters through online channels. And they want to grow that by some number because they’ve obviously figured out that the acquisition cost goes down and people are more loyal when they’re interacting with their app or their website. So if you can align your program behind those things, what we find is people get started, they start to see some success because usually when people get there, there’s a lot of low hanging fruit in there, a lot. And then they start to see things like conversion or acquisition go up. Their business gets more efficient and the kind of thing sort of cell phones inside the company. So a little bit of a diversion from your question. Right. But I definitely think that there’s a lot of opportunities for folks to do this.
Rob Pegoraro [00:14:31] Right. And to what extent have you found that the work you do for various customers and helping them optimize their pitches flows right into your pitches to your various business customers? I’m sorry, to what extent has the lessons you’ve learned in helping your your customers optimize their pitches to their customers? Is that sort of a virtuous circle where you can then turn that around? OK, we know the company of this size, of this level of challenger status.
Carl Tsukahara [00:14:57] Yeah, yeah, that’s a great question. I think one of the things that’s really important in our relationships are really our position in the market is that if you’re just just getting started, we think that best practice is super important. And so the way we kind of look at that is there’s a couple of ways. One is we call maturity level. So, you know, are you a beginner or have you been doing some of this but at low volume or you have a small number of segments? You know, we work at The Gap. They have over 200 different persona segments. So they’re kind of over on the right hand side of this for really figuring out how to bucket and drive intent. But there’s companies just getting started. So, you know, we have to provide some best practice. So, you know, how do other people in media or in transportation or in e-commerce or B2B kind of look at this discipline? And I think that is almost as important as the technology itself, because we can really help guide people through a lot of different issues, like thinking about this strategically. How does this fit into your organizational model? If you have creative and you have a product team and you have development and you know, how how does that work? So. I really think of this as kind of using a little bit of a corny word, it’s a really holistic practice for the business, but I think all of that soft stuff is actually super important because if you do this right, like, you know, we work with IBM and they have thousands of users here, but they started small to you know, they started five or six years ago with us with just a really like we were going to say 100 years ago. Now there’s just, you know, five or six years on the digital side. And so they’ve grown their thing over time. Obviously, they have a lot of resources as a company, but it’s all that practice and how you build this, because it is really you know, we have customers that get 100 Exaro. And that’s pretty amazing for a team to be able to show that your management of saying, hey, you know, we put a million or five million into this program and we’re getting, you know, four hundred million dollars of revenue back out. So, again, I think the soft piece of how you grow that is as important as probably the hard piece in the product.
Rob Pegoraro [00:16:56] So we’ve been talking all along about optimizing and personalization, but do we want to sort of pull back and offer any sort of general advice to startups in this room about mistakes they might be about to make, but they don’t have to?
Carl Tsukahara [00:17:09] Yeah, I mean, I think when we do run into mistakes, you know, if you’re a startup and especially if you’re a cloud only or digital only startup. Right. Or even if you’re a division of a company that’s really chartered to do that or transform part of the business. As far as customer experience, like I would just say that gets started with this discipline early. You know, we have a fairly large client sales, consumer goods, and there’s, you know, sometimes inside the business there’s kind of a war between the design team, development, marketing, e-commerce and trying to figure out like what should we deploy? What’s, you know, how do we figure out what parts of the site to make better know? How do we think about our authenticated mobile experience? Who owns that? But I would say if you can start to bring this discipline in early and, you know, so we’ve worked with companies like Stitch, Fix and Fresh Direct and Kaspar’s and a bunch of companies that started out, you know, just being a super early disruptor brand, but grew up a lot because I think that once you know, that whatever you going to deploy is going to work, it just makes your growth opportunity as a business so much more powerful. But again, this is not just products, so don’t make the mistake of thinking about this as a tactical. I’m just going to do one test on this landing page because that’s where a lot of our paid search comes from. A great I change the image type from here. You know, that’s great. That’s tactical. But if you start to really build this as a culture and you know, I know Jeff Bezos and Reed Hastings, all those guys talk about this has been fundamental to their innovation because you can take chances because you can test them. If you deploy a new ecommerce experience and it blows up your site and your customers leave. If you’re an early stage company, this is a disaster. Right? So if you can start to really think about this discipline early in your life cycle, bring this in. Just make sure your senior leadership is aware that this actually will make you way more agile from a religious perspective, because you can test it at low risk points in your deployment and not have to roll these things out to the entirety of your user population. You can get statistical significance and know if it’s working. But bring this culture, bring this discipline in early. I mean, that’s probably the one bit of advice and kind of help when you’re getting started. Put this as a fundamental piece of your kind of product development and marketing culture.
Rob Pegoraro [00:19:28] Of course, with Amazon, I have to note, there are also some of the more generous retargeting companies. I think back to the time I looked up the price of a toilet seat and Amazon thought I was collecting toilet seats. So I did have to turn off the retargeting option in their ad preferences.
Carl Tsukahara [00:19:46] Yeah, I mean, again, this is I think with with this kind of capability, I mean, this is not a zero or one, you know, you’ve got to decide, like how and this is where the maturity thing that I talked about earlier really matters, because at a lot of it, when you think about where you sit on the maturity curve, some of it is organization maturity and how well you know, how does this fold into your strategy as a business? And then how do you kind of build your program behind that? Because it’s again, it’s it’s not a zero or one like, you know, you’re your team is going to either need convincing or, you know, we went into some organizations where this is just we’re going to do this. We’re gonna do this at scale. Everything we roll out will be tested. But you’ve got to figure that out because, you know, you don’t want to get like we talked about earlier, Rob, you want to get creepy or think that it’s going to diminish your brand. Obviously, you can test those things. But, you know, again, this is just something that I highly, highly recommend. You think about your business early when you’re building out your experiences, because we just seen a lot of these Challenger brands, um, have massive impact on their growth. And it makes sense. You know, you’ve got a lot of people working on your experiences, whether you acquire customers or retain them, you know, over online channels of various types. Get it right, get it right every time. And that’s probably the essence of experimentation is getting kind of predictive outcome data that, you know, as it rolls out to your traffic, will continue to drive the results you want. And, you know, if you think about some of those Challenger brands I mentioned, that’s absolutely essential. If you can make sure that your users will engage, they will like it and they will convert. It sounds super basic, but I’m telling you, just having seen this across thousands of clients, you know, I would just really try and drive this into your business from the get go.
Rob Pegoraro [00:21:32] So we started on the theme of optimization in the terms of the timing of the panel, based on what we’ve seen from the conversation and audience feedback, how would you optimize the title of this panel?
Carl Tsukahara [00:21:42] I think I’d probably optimize the title of the you know, the last question you asked, like, you know, how do you think about this philosophically for your business? You know, it is a strategy. It’s not, um, you know, you think about this as a tool. I think you’re looking at this completely the wrong way. And so maybe optimizing the talk. So I was super close with, you know, I think about like, how should you think about optimization and personalization in your business? You know, it’ll be a tool optimization tool. Don’t be a personalization tool. Underline. Yeah, yeah. That’s that would be probably how I would tweak it just to make sure that I think that’s that maybe is the most important tip is to, you know, really think about this, how it fits into your culture from the get go.
Rob Pegoraro [00:22:23] All right. Thank you very much.
Carl Tsukahara [00:22:25] Thank you.