Subtitles

1158
00:00

The Sean Ellis test, such a seemingly simple idea that has had such a profound  impact on the startup world. The question is, how would you feel if  you could no longer use this product? Once you got a high enough percentage of  users saying they'd be very disappointed, most of those products did pretty well. If you  felt too low, those products tended to suffer. Say someone is listening and  they're like, "Okay. Man,

00:21

I'm getting like 10%. I don't know what  to do." What do you find often works? Just ignore the people who say they'd be  somewhat disappointed. They're telling you it's a nice to have. If you start paying  attention to what your somewhat disappointed users are telling you and then you start tweaking  onboarding and product based on their feedback, maybe you're going to dilute  it for your must have users. Moving retention often is really hard, but I guess  it sounds like there's often something you can do.

00:45

It's usually much more function of onboarding  to the right user experience than it is about the kind of the tactical things that  people try to do to improve retention. What are like three or four things that you think people should definitely try  to help improve activation? In my experience- Today, my guest is Sean Ellis. Sean is one of  the earliest and most influential thinkers and

01:09

operators in the world of growth. He coined  the term growth hacking, invented the ICE prioritization framework, was one of the earliest  people to use freemium as a growth strategy, and maybe most famously developed the Sean Ellis  test to help you understand if you have product market fit, which a large percentage of founders  use today and profoundly impacted the way startups are built. Over the course of his career, Sean  was head of growth at Dropbox and Eventbrite,

01:33

helped companies like Microsoft and Newbank  refine their growth strategy, was on the founding team of LogMeIn, which eventually sold  for over $4 billion, and he's the author of one of the most popular growth books of all time  called Hacking Growth. In our conversation, we dive deep into two topics. One, how to know  if you've got product market fit and what to do if you don't, and two, how to figure out how  to grow once you've found product market fit.

01:58

If you're in the early stages of a new  product wrangling with product market fit or trying to figure out how to jumpstart or  further accelerate growth for your product, this episode is for you. If you enjoy  this podcast, don't forget to subscribe and follow it in your favorite podcasting  app or YouTube. It's the best way to avoid missing feature episodes and it helps the podcast  tremendously. With that, I bring you Sean Ellis.

02:23

Sean, thank you so much for being  here and welcome to the podcast. Thanks, Lenny. I'm super  excited to be on with you. There's so much that I want to talk about.  There's so many directions we can go, but to keep it focused, I want to spend  time on two areas. I want to talk about how to know if you have product market  fit and what to do once you have product market fit in terms of figuring out how  to grow. I know these things are very

02:45

linked. I know you spent a lot of time  on these things. How does this feel? Sounds perfect. Yeah, let's do it. Okay. Okay, amazing. Let's talk about,  first of all, the Sean Ellis test, slash something people call  sometimes the product fit test. Such a seemingly simple idea that has had such a  profound impact on the startup world. I've never

03:07

actually seen you talk about the history of this  thing, how you came up with these questions, how you came up 40%, the whole journey of this thing.  So let's talk about this. But first of all, can you just tell people what is the Sean Ellis test  for folks that aren't exactly familiar with this? It's a simple question that helps you figure out,  does anyone consider your product a must-have, or ideally, who and how many people consider it,  but ultimately it's about trying to figure out is

03:31

your product a must-have, which could be equated  to having product market fit. And so the question is, how would you feel if you could no longer  use this product? And I give them the choice, very disappointed, somewhat disappointed,  or even not disappointed or not applicable, I've already stopped using the product. And what  I'm trying to find are those people who say,

03:54

"I would be very disappointed if I could  no longer use this product," then that's a really powerful vein to dig  into when you discover that you actually have some people who would  give a crap if your product disappears. This episode is brought to you by Gamma,  an entirely new way to present your ideas powered by AI. If you hate designing slides and  dread that feeling of staring at a blank slide,

04:20

Gamma is here to help. Just upload your PRD  and turn it into a beautiful ready-to-present presentation in seconds. Gamma works with  all types of formats from Google Docs, PDFs, to PowerPoint. You can even drop in a  link to your favorite Lenny's newsletter post and turn it into a presentation for your team. Gamma has become one of the fastest growing AI

04:41

web products in the world, adding 20 million  new users just this past year and is setting its sights on becoming the modern alternative to  PowerPoint. Whether you have design skills or not, Gamma can save you hours of time synthesizing  your ideas and shaping your content. Visit gamma.app and use promo code LENNY to get a  free month of Gamma Pro. That's G-A-M-M-A, .app.

05:07

Let me tell you about CommandBar. If you're  like me and most users I've built product for, you probably find those little in-product  pop-ups really annoying. "Want to take a tour? Check out this new feature." And these  pop-ups are becoming less and less effective since most users don't read what they say. They  just want to close them as soon as possible. But every product builder knows that users need help  to learn the ins and outs of your product. We

05:29

use so many products every day and we can't  possibly know the ins and outs of everyone. CommandBar is an AI-powered toolkit for product,  growth, marketing and customer teams to help users get the most out of your product without annoying  them. They use AI to get closer to user intent, so they have search and chat products that  users describe what they're trying to do in their own words and then see personalized  results like customer walkthroughs or actions.

05:53

And they do pop-ups too, but their nudges are  based on in-product like confusion or intent classification, which makes them much less  annoying and much more impactful. This works for web apps, mobile apps, and websites. And they  work with industry-leading companies like Gusto, Freshworks, HashiCorp and LaunchDarkly. Over  15 million end-users have interacted with

06:14

CommandBar. To try out CommandBar, you can  sign up at commandbar.com/LENNY and you can unlock an extra 1,000 AI responses per month  for any plan. That's commandbar.com/LENNY. And the idea is that if you, 40% or more  of people, say they'd be very disappointed

06:36

if they can no longer use the product,  you essentially have product market fit. I would say it's a leading indicator of product  market fit. The lightning indicator is, do they actually keep using it? So probably retention  cohorts are more accurate, but the problem is, like your time at Airbnb, how long do you have  to look at a retention cohort before you know

06:58

that you've actually long-term retained someone? And so with this question, you can kind of find out day one, you don't need a good analytics  system in place to be able to see if product market fit exists. And so yeah, the 40% was not  something I originally had in there. Originally,

07:21

I was trying to have just a filter so that I was  not treating all feedback from customers the same, but I was trying to find feedback from customers  who actually really cared about the product. And then was over time, at the time I was  working for a couple of YC-backed companies, and so those companies were all pretty connected,  and so I would share the question with a lot of

07:46

other startups in Silicon Valley. And so over  time, I started to see there was a pattern that once you got a high enough percentage of  users saying they'd be very disappointed, most of those very disappointed without  the product, most of those products did pretty well. And then if you felt too  low, those products tended to suffer. Okay, there's two things I want to  definitely follow up on here. The

08:07

first is such an important point that you made  at the beginning when I introduced this test that you described it as a leading indicator  of product market fit and actually retention, people actually using your product, the  product actually being used by the market is the actual ultimate test. So the idea  here is this is a good way to get a sense of, before you actually have data, are we headed  in a good direction? Could you speak more

08:29

about that, of like, when to use this and when  it's most useful in best [inaudible 00:08:34]? Yeah. I mean, so for me in particular,  when I come into a company, my goal is to help them grow. And so I don't want to put  myself in a situation where I'm going to fail because no one actually cares about the  product. And so it can really be asked

08:49

at a company of any stage. It's helpful  to understand who your must have users are. But essentially once you have even an  MVP, like a very first MVP on the product, you can still get some useful feedback about  the product if it's resonating with anyone. So I actually had a company where I had  committed to work with them. It was right

09:14

after I left Dropbox and I committed to work with  these guys for six months to help them grow. I ran the question and it came back at only 7% of  users saying they'd be very disappointed without the product. And so I'm like, "I have six months  to help them grow and they're only at 7% right now. It might take six months to get the 40%. Am  I doing them a disservice by being in a growth

09:36

role and being on payroll during this period  of time?" But fortunately with the signal and the information we got from the initial survey,  we were able to get them at 40% in two weeks. Wow. What did you do there just as a case study? Yeah. Yeah. So the company called  Lookout, it's a mobile security company,

09:57

and now most of the things in Lookout are built  into iPhones and Androids. But at that time, the product had everything from backup my data to  find my lost phone to protecting your phone with a firewall and antivirus. And so when we ran this  initial survey, I dug into the 7% who said they'd

10:20

be very disappointed without the product and found  that most of that 7% were focused on the antivirus functionality. So they were like, they know they  need to protect their computer from viruses, smartphones were becoming more like computers, so  it just made a lot of sense for them that they'd need to protect their phone. And interesting,  at the time, I think there was only one kind of

10:44

phone virus that had ever even happened, but  it was a pretty easy mental leap for people. And so now we knew, okay, it's antivirus that  people really valued. And so step one was just reposition the product on antivirus. So that  kind of creates a filter. So anyone who now is coming in to sign up for the product who doesn't  care about antivirus is not going to convert, and

11:09

those who are excited about antivirus are going to  convert. We already know from the initial survey that people value that after they convert. So by  setting the right expectations around it up front, you're going to bring people in with the right  expectations. But then the second thing that we did was we streamlined onboarding so that the  first thing that they did after signing up for

11:32

the product was to set up the antivirus and then  get a message "you're now protected from viruses." And so it's really the combination of those  two things. It's set the right expectations and then speed to value. And so the next cohort  of people that we surveyed were at 40% saying they'd be very disappointed without the product.  So that literally took two weeks to make those

11:53

changes. Six months later, it was 60% on the  score. And then I think they hit the billion dollar valuation four or five years later on  ultimately being one of the early unicorns. And interestingly, as all of those things were  built into mobile phones now, they've completely

12:17

changed the business, but they continue to do  really well, but they've continued to iterate the business. I think that having that kind of  finger on the pulse early in the business was important to build the muscle in the business  to be really responsive as the market changed. Sean, this is already amazing. There's  just a fractal of topics I want to

12:38

explore from this very short conversation  already. So the first is just follow this thread of basically you're sharing kind of a  growth strategy that I imagine you execute, is look for the percentage of people that would  be very disappointed if your product went away, see who they are, see what they're excited  about and lean into that both positioning-wise, onboarding-wise, and probably also cut out stuff  from your product that they don't care about.

13:02

Yeah. And I was coming at it from a marketing  perspective initially. Over time, I position myself more in a growth role with product  and marketing as areas I could influence. But as a marketer, I probably didn't have a lot of  influence on a engineering founded company to say,

13:22

"Let's cut out stuff." So it made more sense  to say, "Let's just sequence the onboarding so that we're highlighting this and onboarding  to this." That was a little easier to sell. And just hearing that you can move this score  so quickly without even changing the product substantially, I imagine what surprised a lot of  people when you think about moving retention often

13:46

is really hard. And maybe we talk about that, but  I guess it sounds like there's often something you can do that's not very hard that might  significantly shift this product market fit test. Right. And then that ultimately  moving retention is really hard, but it's usually much more function of  onboarding to the right user experience

14:06

than it is about the tactical things that  people try to do to improve retention. Okay. I want to put a pin on that and come back  to that because a really important topic. I'm going to come back to, say someone  runs this survey and they get 40%, what should they have in their mind of like,  "This is what this is telling me"? Because I think a lot of people are like, "I got  product market fit. I got this. Let's go,

14:27

go, go." What's the best way to  think about what this tells you? Yeah, I mean it tells you something really  important, which is, you haven't created something that people don't care about. So  that's an important insight. But until you deeply understand that product market fit, you  kind of don't have the tools to be able to grow

14:47

the business. So that's really the next step, is  to dig in and figure out who considers it a must have, how are they using the product, what did  they use before, what problem are they solving. One of my favorite questions is... So I tend to  have a lot of questions that I build off of that I'm using that filter, trying to drill into the  users who say they'd be very disappointed without

15:12

the product, and one of my favorite questions  is, "What is the primary benefit that you get?" And then I use that initially as an open-ended  question to kind of crowdsource different benefits people are getting. But then I run another survey  where I turn it into a multiple choice question, force them to pick one of four distinctive  benefit statements. And then the question

15:36

that follows on that next survey is,  "Why is that benefit important to you?" And then I start to get really good context. So I actually came up with this question when I was working with an early YC company called Xobni,  which is inbox felt backwards. And when I ran that

15:56

question, basically the people who said they'd  be very disappointed without the product we're focused on, "Xobni helps me find things faster in  my email." So it's great to know, okay, that's the benefit. But when I asked, "Why is that benefit  important to you?" They said, "Oh, I'm drowning in email." I kept seeing that statement as a written  statement. And so when I then was trying to figure

16:20

out how to acquire customers, when I tested  "drowning in email?", that set such a good hook. That was the context that people were living in,  that they were really responsive to the message of find things faster with Xobni and then a  description of what Xobni is. So I think when you can really dig into the context of why that must  have benefit is important to people, you start

16:45

to get the ingredients to build that flywheel  that leads to long-term sustainable growth. So what I'm hearing is whether you have  40%, whether you have 60% or even 7%, the actual best use of this tool  is look at that percentage of highly disappointed and see what they're  looking for, what they're excited about. Start drilling in, start feeling back that onion-

17:05

Start drilling in. ... and just deeply understand them and make  sure that ultimately your product roadmap is doubling down on the things that are important to  your must-have customers. Onboarding is bringing new people to the right experience. Your  messaging is setting the right expectations, your acquisition campaigns are targeting  people who actually have the need.

17:29

And so it's all about getting the right people  to the right experience. And then even your engagement loop is about just reinforcing how to  get people to experience that benefit more often. Awesome. And the 40% threshold, so  what you shared is you basically emerged from just looking at tons of  startups doing the survey and finding a pattern. How firm is that 40%? How  big of a deal? Is it 39 versus 41?

17:52

I don't think it's that firm. To me, I think the  real power is having some kind of target for the team to be shooting for that basically says,  "We're not going to aggressively start to grow until we hit this target." And I think that  as just a focusing piece is really important because I think one of the biggest challenges  in an early stage startup is half the people

18:17

feel like we're years from having this product  ready to grow, and half the people are like, "What are we waiting for?" Where if you can  actually get people on the same page of what does product market fit look like for our business,  and it's at that point that we're going to. Before I ever heard the term product market fit,  I remember the conversations back at LogMeIn in

18:41

the mid 2000s of kind of like, "When do we  step on the gas? What is the combination of factors that need to be in place before we start  pouring fuel on the early fire?" And so yeah, I think that kind of nail it then scale it.  It's probably been a term that's been around for decades now, but it's all kind of pointing  to that same concept of product market fit.

19:03

How often have you seen false positives  with this test where someone gets 40% and something is not right, they're actually far  from it? Or is it generally pretty accurate? If you're having people say that they'd  be very disappointed without your product, that's a really good sign. What I can tell  you is that not necessarily a false positive,

19:25

but what is driving people to say they'd be very  disappointed. One of my favorite books is Hooked by Nir Eyal and he talks about in the kind of  engagement loop that your last step is investment. And so I ran the survey on a business that I  thought was a fairly commoditized business.

19:46

Part of it I wanted to see, could I use the  same go-to-market approach on a later stage company and use it to accelerate growth.  And so this was a business called webs.com. They eventually got acquired by VistaPrint. But  they'd been pretty flat for the year before I went in there. And then I started to use this approach  to try to dial in their growth engine. I ran the

20:12

survey thinking, "Yeah, you've had products like  Wix and Weebly that have come on to the market since this more legacy website building product  has been around. I personally think they're a little easier to use, they're a little better."  And so I didn't have high hopes when I ran the survey, but it came back with one of the highest  scores I'd ever seen. And it was like 90% of the

20:35

people saying they'd be very disappointed  if they could no longer use the product. Holy shit. I've never seen that. And I was like, "How could that be possible?  This product is kind of a commoditized category. I wouldn't even say it's one of the best."  And then when I want to dug into it again, it comes back to that Nir Eyal Hooked model, is  that the investment people have made in building that website, they put so much into that they know  exactly how to make the changes and the kind of

21:01

the CMS kind of side of things, they have spent  a lot of time just making it beautiful. And so ultimately it was something that that was why  they were saying they'd be very disappointed. But fast-forward when I initially went in,  still doing these things help the business

21:24

resume growth and have significant growth  over the next 12 months after we did these things. So still the signal we got from why  people would be very disappointed without the product was important and speed to value. All  the other things I think about in go-to-market for an early stage product still were relevant,  but just I think they were a little stronger on

21:48

the percentage he'd be very disappointed. Even Eventbrite when I was there when we ran it was probably the second highest  I'd ever seen. But with event organizers, if they've already set their event up on  that platform and they've sent it out to their list and all those people are coming  in and they're managing their event, again,

22:10

they've invested a lot in the platform.  So sort of switching costs I think can factor in there. So it's a function of both  switching costs and utility of the product. So that's a question I wanted to ask is, what's your guidance on when to ask this  question? What I'm hearing is, if you ask it very far along the journey, when they're very  invested, you'll get a much higher score. Is

22:32

there any advice on the timing and the best  time to ask this question to your users? What I recommend is a random sample of  people who've really used your product. So they've gone in, they didn't just sign  up, but they went in and hopefully hit that deviation moment. They've used it twice, two  plus times, and they've ideally used it, say,

22:58

within the last week or two weeks,  so they haven't churned yet. So if it's a random sample of those people,  that's kind of the ideal time to ask it. Got it. So basically it's people that  have activated whatever that means to you and have been using it for a couple weeks? Yeah. Not people landing in your home page, not  people just signing up, not people months later.

23:19

Not people who've seen a demo of your product,  but it's people who actually have experienced the product. But it's okay if you're hitting people  who've used it months later, but in that Lookout example that I gave, if I'm testing people's  perception of the product after I made updates to the onboarding, I'm going to only want to  survey people who went through the new onboarding.

23:43

Yep. In the experimental. Yeah. Okay.  So I asked people on Twitter what to ask you. A lot of people had a lot  of awesome questions. I'm going to sprinkle in a couple of these  questions throughout the chat. Sure. One came in from Shraaz Doshi,  a popular guest of the podcast. One of the ones that I listened to recently. Amazing. I think it's the second most  popular episode behind Brian Chesky.

24:05

Okay. So he had a question of just, "What  are the limitations of the score? When does it break down? When should you not use it  if ever?" Is there anything of just like, "Here's when it's not going to work for you"? Yeah. I think one-off products would probably,  like, how would you feel if you could no longer watch the movie you just watch? I  wouldn't care. Even when I run a workshop, I don't run this as part of my survey after I do  a workshop because how would you feel if you could

24:31

no longer attend the workshop you just attended?  It doesn't make sense. So I'll ask an NPS question as my filtering question so that I'm looking at  focusing in on feedback of people who love it, also then through a separate lens, looking at  people maybe who would be my detractors. So I think one-off products are probably not good  products to run the question on. There may

24:55

be other places as well that I'm not thinking  of right now, but that [inaudible 00:25:00]. It sounds like not many. What I'm hearing  is it's generally widely applicable. Yeah, I think it is, at least from my perspective.  It's been really useful for me anyway. Awesome. Okay. And then the follow-up question  from Shraaz is, and I kind of asked this,

25:15

but I'm curious if there's anything more  here, just, "Have you seen any instances of startups over relying on the score prematurely  declaring product market fit when in reality they haven't reached it yet? And just are there  any other caveats of like, 'Cool, I got 40%'?" Is there anything else you should know, like,  " Okay. But maybe check this one thing"? Yeah, I mean, I think to me it's kind  of like what really is the definition

25:35

of product market fit is the definition that  people who get through my crappy onboarding and actually experience the product love  it. And if I'm able to retain those people, that means I have product market fit.  Or, is fixing that crappy onboarding part of getting to product market fit  as well? I think that's up for debate. So to me, the hardest, I wouldn't obsess on  onboarding if I know those who kind of get

26:04

through the challenge of getting started with the  product still don't like the product then feels like it's a core product issue or wrong people  using it in the wrong way issue. But once you have that, then ultimately it doesn't mean that  you're ready to grow. When I focus on growth, then customer acquisition is almost the last  step. Once I validate that it's a must have

26:28

for those early users, then I'm thinking about,  "Okay, how do I optimize speed to value? How do I make sure that people have the right prompts to  come back and use the product at the right time so that's kind of more of that engagement loop? How  do I get my existing users to bring in more users if there's something that makes sense on that  end? Even how do I optimize my revenue model?"

26:52

Once all of those things are working well,  then I'll obsess on the customer acquisition side. But customer acquisition is so hard that  if you're not really efficient at converting- ... customer acquisition is so hard that if  you're not really efficient at converting and retaining and monetizing people, you're going to  really struggle on the customer acquisition side. Yeah, cool. And we'll talk about  customer acquisition/growth.

27:13

Sure. Another question I wanted to ask, and  a couple listeners asked, is the 40%. I had Jag from Nubank on the podcast,  I think you may have worked with them, and they use 50% as their threshold because  apparently Brazilians are very nice. Yeah. Optimistic I think is what I said. Yeah. I guess the question is  do you find instances where you

27:34

should increase that percentage? And  in B2B, is anything different? Do you change the percentage in B2B? Any advice  there just when you adjust the threshold? Yeah, I hadn't really thought too much on that.  Again, for me, generally I'm trying to just figure out is this a product that can grow? So  if I got a 37%, am I going to be like, "Oh no,

27:59

this would be impossible?" Or if I had a 70%, does  that mean I'd say, "Oh yeah, I want to jump in and work with this company?" It's more nuanced than  that. Obviously, if it's a 70%, but I have no idea how I grow the business, I'm going to be stuck  there. But I do think he brought up a really good

28:23

point that, culturally, some people are going to  be more optimistic or pessimistic. Interestingly, when I came up with the question, I used  to just use a normal satisfaction question. When I was working at Xobni, I'm just an intensely  curious person anyway, so I'm just trying to dig

28:45

in and understand the customers, and so I've  always done lots of surveying. But at Xobni, I was going to use my filter as a satisfaction  question, so how satisfied are you with this? I'm very satisfied. I'm somewhat satisfied. And our  main customers were actually senior management, and so I thought senior management's never  satisfied. I'm going to get always this super

29:09

lukewarm thing. How can I change this question to  give me a more real answer from these guys? Well, if I flip it and say, "How would you feel if  you could no longer use this product?" I'll probably get a more honest answer back  from them. And of course, they're very disappointed if they can't get what they want. And so initially it was just for the case of

29:32

Xobni, but then I went to Dropbox right after  Xobni and like, "Oh, I'll try the question again." And the insights I got back were really useful.  And so each company I went to, I kept using the question. I'm like, this works way better than  your typical satisfaction question. But initially, it was more about thinking just senior management  to get a more honest answer out of them.

29:54

So that's the origin story right there? Yeah. Wow. That senior managers are just very  harsh and they don't need anything? Yeah. And you have to flip it. That is so  interesting. That question is such a good reminder of how hard it is to  build anything people really would be disappointed not to have. That's why  this works so well. People are like,

30:14

"I don't need this. Who cares?" That's  the core of this, is just that is hard. Especially when I first moved to Silicon  Valley. The first 15 years of my career were not in Silicon Valley, and so I was  in Eastern Europe and then New York and then Boston. But you move to Silicon Valley  and you have people who get really excited about technology for technology's sake.  And so just something being cool is like,

30:40

"Isn't it cool that we can actually do  this?" drives a lot of people. And so to me, I'm very practical. If it's not something  that is really bringing value to people, then the likelihood that that product's  successful long-term is going to be pretty low. Even, interestingly, at Dropbox, through the six  months I was there, I'd ask one question multiple

31:07

times a month. I broke the early beta users  into a bunch of different lists. And I'd ask, "Which best describes you? I like to be among  the first to try cool new technology, or, I only try things that I think will be useful for  me?" And over the six months, it flipped from 90% being people who try things that they want to  try cool new technology to six months later,

31:31

it was people who only are going to try something  that they feel like is useful. But what's cool is just because what motivates you to try something  is you're an early adopter and you want to try something cool, if you're going to keep using it,  it's because it's giving you some utility. And so I can still use those early adopters to help me  figure out where's the value inside the product.

31:54

Awesome. So actually, two questions along those  lines. How durable do you find this percentage being? Say you hit 40%, how often does that  fade and go away versus stay there or go higher? Yeah, I haven't seen it really fade back down, but  I've seen companies fail despite having it. And I

32:14

think a lot of times then, it becomes an execution  challenge. Once you have product-market fit, not everyone's going to be a good executor. But  before that, I think getting to product-market fit, obviously there's a lot of methodology  for doing it today that might make it a bit

32:36

easier for people, but I still think it's fairly  random and pretty dang hard. And so ultimately, the risk factor of creating something that people  care about is really difficult. So if you can get to the point where you have 40% of the people who  are using it saying they'd be very disappointed, and you have a reasonable sample size. Let's  say you've got 10 people and four of them said

32:56

they'd be very disappointed without it, you're  still going to get something useful from those four. But I wouldn't say that's a sample size  that you can really go to market on, so yeah. What's a good sample size  you look for of just, "Okay, this is actually good data I want to rely on?" It's really funny. So much of the stuff I  self-learned, but I basically at one point said

33:17

I need at least 30 responses, and I just thought  I randomly made up a number and then I had people telling me, "Yeah, 30 is the minimum that you want  on stuff." Okay. And even when I first created this survey, I remember showing it to the  co-founder of SlideShare and her PhD was in survey-related stuff like cognitive psychology,  but she basically said it was really about

33:41

surveying. And she's like, "This methodology  is amazing. How did you come up with this?" And so having some of that validation around  these things helped. But a lot of it was just, again, driven by my own curiosity and also  just knowing that that failure is such a likely outcome that trying to reverse engineer  that failure, and then the number one reason

34:05

for failure would be that people don't actually  care about the product. And so when I find that, that's a really good sign that we're  now down to an execution challenge. And there's this obvious element of you  may have product-market fit with people, but that group might end up being very small  and the business you build around it could actually be cool, but it's not going to be a  massive business. Is there anything there you

34:28

can share? It's hard to know the size opportunity  even though some people really, really like it. Yeah, I talked about I go to a multiple choice  after I initially use open-ended questions to crowdsource the different use cases. But  then I try to force people in a bucket, and then I can run filters on each of  those buckets and I'll be like, "Oh,

34:51

people who use it this way are like 60% likely  to be very disappointed without the product, but people who use it this way are  35% likely to be very disappointed, but way more people use it the 35% way."  And so then, do you want that intensely loyal group or the much broader group  that's maybe a bit less, but almost there?

35:17

I think that becomes a bit of a strategic  conversation of do we want to have a better chance of surviving, going after a niche that  we know we can serve well? Or have we raised so much money that we have to go after a really big  market, and one that's not going to be long-term?

35:38

But maybe then you're like, "Okay, once I have  traction in that market, I can start to try to appeal to some other markets." But I think  that's where some strategic decisions come in. Do you have a heuristic of  which you often recommend or is it very dependent on the situation? I prefer a more passionate customer base and  work from there, just because I think your

35:59

biggest competition when you're really innovating  is just being irrelevant. And so if you're deeply relevant to anyone, I think that gives you  a much better chance of long-term success. Awesome. That's a really good insight. Okay,  two more questions along this line and then I want to talk about growth strategy. One very  tactical question. Is there a tool you recommend

36:20

for doing this sort of survey? Do you recommend  inline? In the product? An email? Something else? I've used a lot of different tools. I actually had  a survey business that I sold to private equity years ago. It is called Qualaroo. That's an inflow  survey tool. I think just using SurveyMonkey with

36:47

emailed surveys works fine. And for me, it's a lot  more of what's pleasant for the customer to fill out and then what's going to give me something  where I can work really easily with the data? So at Bounce, for example, they had already intercom  in place that had just introduced surveying,

37:08

but it was a really crappy customer experience,  at least at that time. That's been almost a year now or actually a little over a year. And so  I'm really sensitive to is it a good survey experience for the consumer itself? But yeah, I  don't think I'm stuck to any one platform there.

37:30

Such an important topic. Just, again, to  remind people why this is so important, one of the most common questions founders ask  is, "Do I have product-market fit? Have I built something people want?" That's just an endless  series of, "I don't know. How do I know? When do I know?" And this is telling you in a really  interesting way. So your advice is this is a leading indicator. You don't actually know until  people actually start using it and whether they

37:52

retain and continue using it. Is there just  advice on the shift you make from relying on the survey to actually looking at retention  cohorts? Is it just once you have enough data, once you have a couple of cohorts, then start  looking at that? Forget about the survey? Yeah, but retention cohorts don't give you  any of the qualitative insights into the why, so that's why we continue to do the survey. So  initially I would say if the survey comes back

38:18

and it shows whatever your target number is...  If you want to be Nubank, it'd be a 50%. Or two of the companies I launched, we launched in  Hungary, and I would say it was the opposite end of the spectrum of Brazil, maybe more pessimistic  than the average culture. And so maybe 30% is good

38:40

enough there, but that ultimately, whatever your  target is, that you have the signal that says, "Okay, we have enough value here. Let's start  working on growing the business." But while you're working on growing the business, I  would be paying attention to those retention cohorts. And if you're churning out all the  customers who were saying that they'd be

39:02

very disappointed without the product,  then okay, let's retrench and rethink, do we really have product-market fit here and  what do we need to do to get it if we don't? Awesome. And speaking of Nubank, if anyone  wants to see how a company has actually operationalized this in the way they operate,  that there's an episode that we'll link to in the show notes where every new product at  Nubank they build, before they launch it,

39:26

they wait for 50% threshold. For people to  say 50% of people would be disappointed if this product did not exist as they're developing  it. And only then do they launch it publicly. Yeah, I think they even do  it down to the feature level. Wow. So if you think about it, how would you  feel if you can no longer use this feature starts to give you, again, the signal, is that  feature a must-have feature? And if it's not,

39:48

maybe we shouldn't have it. And so yeah,  I was super excited when I saw how they were using the survey and they were  doing it before I engaged with them. Oh, wow. That's awesome. But they were doing it, I think,  from pretty early on in the business. The reason they can do this is they have a lot of  users. They have millions of millions of users,

40:09

so they can ask some small percentage  of people this question. Because people hearing this might be like, "Oh my God,  how many times am I going to be asked this question when I'm using this feature?"  But they have a lot of users, so it's easier. Yeah. Yeah. Okay. Last question, I promise, along these  lines. Say someone is listening and they're like, "Okay. Man, I'm getting 10%, I'm getting  15%. I don't know what to do to increase

40:29

my product-market fit." You should have just  a strategy of just dig into the people that are very disappointed and see what they  have to say. But any other advice/what do you find often works in helping  people move from, say, 10% to 40%? Yeah, so one of the things that's cool about  almost open-sourcing the survey approach is,

40:50

again, watching how Nubank has evolved their  usage. But one of the other companies that I think used it in an interesting way is Superhuman. And  I would say that they basically ended up probably putting a lot more momentum behind the question  than it had even before. They posted something about how they did it on First Round Capital's  blog. And what I have always said, and again,

41:15

it's me coming at it from probably initially a  marketing background, which is I'm taking the product as a fixed thing, and how do I actually  figure out how to market and grow this product? And product changes are going to take a long  time, and so what are the variables that I can control with a marketing background? So one of  the things I've always said is just ignore the

41:42

people who say they'd be somewhat disappointed.  They're telling you it's a nice to have. They're as good as gone, so just ignore those guys. I'll put one piece in the middle there before I say what Superhuman did. The reason that  I say ignore those guys is that if you start paying attention to what you somewhat disappointed  users are telling you, and then you start tweaking

42:06

onboarding and product based on their feedback,  maybe you're going to dilute it for your must-have users. And ultimately, it becomes kind of good  for everyone but not great for anyone. And so that was my fear of trying to read too much into  the users who say they'd be somewhat disappointed. But the Superhuman guys actually found, I think,  a good way around that where they said, "Okay,

42:28

what is the benefit that my must-have users are  focused on? And then of the users who say they'd be somewhat disappointed, so the nice-to-have  users, of those users who are also focused on that benefit, what do they need in the product  for it then to become a must-have for them?"

42:48

And so they're staying true to that core  benefit, but they're trying to essentially take those on-the-fence users and moving them  up. And so I think their way of approaching that addressed what my concern was, which is are  we going to break it for the must-have users? That's an awesome insight. By the way,  did Rahul and the team there just do this

43:08

on their own or were you involved  in any way in this at Superhuman? No. That's the same thing. Like I  said, I wasn't initially involved with Nubank. I wasn't involved with them.  We wrote about it in our book in 2017, and so I think that I got it out there. But I  actually teamed up with the Kissmetrics team in

43:30

2012, and essentially published this survey on  survey.io where we just made it freely available for people and a really easy template to prepare  and send out, and the how-to guide on it. It was all just free. Kissmetrics is using it as  maybe lead gen. And for me, I just wanted a way to put something out for the community.  And so it's been out there for a long time,

43:55

so it's not surprising that different companies  have found different unique ways to use it. That's awesome. I think that post  is one of the most popular in First Round. It really had an impact on a lot of people. Yeah. So just to repeat, the approach you recommend  for when you're digging into... I wrote this down. When you were talking for how to  dig into what benefit people are finding,

44:15

your advice is it's basically a follow-up survey  to the extremely disappointed people asking them what is the primary benefit you get? It's an open  text initially. Then once you get a collection, you do it sounds like another survey as  multiple choice. Here's five benefits- To a different group of people, to be clear. Different group. Yeah. Got it.  Awesome. And then it's like,

44:35

which of these four or five benefits is  what you're getting out of this product? And then the question is, why is  this benefit important to you? Eventually the survey.io got closed down, but  essentially the template that I typically used was then moved to PMFsurvey.com. And so you'll see  some other questions that I have on there as well,

44:58

like what would you use instead if this product  were no longer available? And that's one of the interesting things is you start to see people  who say they'd be somewhat disappointed, usually, they're focused on a commodity  use case and they know an easy alternative to switch to. So to be a must-have, it  needs to be both valuable and unique. Okay. Anything else on this  topic of the Sean Ellis task

45:21

product-market fit test before we  move on to growth strategy advice? No, I think that's it. I think we did almost an hour on that  one topic, which I love because I feel like this is such a powerful tool that I  think people sort of know and have used, but I think there's a lot of opportunity to  use it more effectively. And all the stuff you pointed out about it's not just you have this  threshold goal, let's move, let's grow. It's like,

45:43

this is how you figure out how to make it better  and better and grow faster and faster. And it's actually a good segue to talking about growth. Even though you coined the term growth hacking, you spend most of your time on the opposite,  essentially, which is helping companies figure out sustainable growth strategies, not just  a bunch of hacks to grow for a little bit and then disappear. And from what I've seen, it's  all rooted in this idea of product-market fit

46:06

and what helps you find product-market fit, and  I imagine many of the stuff we've talked about. Yeah. Just one quick interjection there  is that when I coined growth hacking, I did not think of it as a bunch of one-off hacks.  What I thought of it was what's more about what is the way to ultimately drive sustainable  growth? But it's, over time, maybe more

46:27

interpreted the way you described  it, but just to jump in and say that. That's a really good clarification, so how did  you actually initially frame it when you first- Yeah, I just said it's about looking at every  single thing that you're doing and scrutinizing its impact on growth in the business. And  particularly, I think most marketers, when I first

46:48

moved to Silicon Valley, most CEOs who were asking  me to help their companies, they were saying, "We need help with awareness-building," and I'm  getting introductions from top VCs. And so, so much of, I think, the way people were approaching  growth was marketing textbook how to approach it. And startups just don't have the luxury to do all  of those things, and so you got to really focus on

47:13

how do I acquire customers to an experience that's  going to make them keep using this product? And so maybe I picked the wrong term in calling it  growth hacking, but I think it at least opened the conversation to getting more people thinking  about maybe we should be thinking about growth in a different way than as it's traditionally  taught in marketing courses in school. Is there another term you think you should  have used? Do you always think back,

47:36

I should have called it this? Is there  anything that you've had in your mind? I don't. I think sometimes having something  that's a little divisive is almost better because it's too easy to just go completely  unnoticed. But I was trying to put a name on not just how I was approaching growth, but  seeing Facebook obviously had a very different

47:59

approach to growth than most companies.  LinkedIn, Twitter, there was a handful of companies that were approaching it in the  same way I had previously been approaching it, and I just thought this thing needs a name. And so  I sat down with a couple of friends, came up with a name and it stuck. But yeah, obviously from day  one it was pretty divisive with different groups.

48:23

That's a fun story. Thanks for sharing that. Okay, so talking about growth and helping companies figure out how to grow. Say you go  to a company, they're getting 42% on the Sean Ellis test, and they're like, "Okay, cool, let's  start thinking about growth." What's your first piece of advice to them to start when they're  thinking about growth? And then just broadly, how do you approach helping  them figure out how to grow?

48:45

Ultimately, it's about trying to get as many of  the right people to that same state that we just talked about with the must-have users, so trying  to get as many people to experience the product in a way where they'd be very disappointed  if they could no longer use the product. And so that's not just acquisition, which is  how most companies think about... Initially,

49:07

it was awareness then maybe the more developed  way was, oh, let's at least focus on profitable acquisition. But in my experience, the hardest  part really sits inside the product team, so how do you shape that first user experience so they  actually use it in the right way and it's not so

49:28

difficult that they give up? And that ultimately,  we understand what makes it a must-have product. And then what we're trying to do is build  a... Yeah, it sounds kind of theoretical here, but I can go into the details on how, but  build a flywheel around that must-have value. So step one would be understand it. Step two for  me is then figure out a metric that essentially

49:55

captures units of that value being delivered. And  so when I think about a north star metric, that's what I'm thinking about is something that reflects  how many people are coming in and experiencing that product-market fit experience, whatever  that is. And it's not just me telling them,

50:16

"Here's what your north star metric should be."  It's that ultimately the team needs to decide that together. And then really just diagramming, what  are all of the different ways that we can grow that north star metric? So that's where you start  to actually build, I call it a value delivery engine, but it's what does our onboarding look  like? What's that aha moment? That activation?

50:42

What does the engagement loop look like? Is there  any referral? Try to capture it as it is today. And then, from there, thinking about where are the  biggest opportunities for improvement, so those high-leverage opportunities, and then ultimately  starting to run experiments against those opportunities. Generally, I think I touched on it  a little bit earlier, but generally the sequence

51:03

that I like to do is start with activation because  that one's just so critical and it's easy to get lost in between, especially for an early product.  The product team's so focused on the roadmap. We're two features away from not even needing  marketing anymore. This thing's going to take off. And then a marketing team so focused on bringing  new people in, but how do you get those new people

51:27

to a great first experience falls through the  cracks a lot of times. So a lot of focus on activation and then engagement and referral  and getting the revenue model right. And then once each of those pieces are working well, then  starting to really obsess on the channel side.

51:49

One thing that I'll say. When I go in  and directly am involved with a company on the acquisition side, I am thinking about my  hypotheses on the acquisition pretty early on, because if I go into it and I have no  idea how we'll acquire those customers, I'm not real confident I'll figure it out when I'm  there. So I want to have two or three things that

52:11

seem pretty viable as ways to profitably acquire  customers, and knowing that once I get deep into it, I'll probably come up with one or two more  and I've got five, one of them's likely to work. But I don't want to just be under the pressure  of having to come up with that once I come in, if I don't at least see an angle from that  before I get involved with the company.

52:36

What I'm hearing is when you come into  a company and they're asking, "Sean, how do we figure out how to grow this thing?" you  actually focus first on activation onboarding, and we're going to talk about all these things.  Then, after that, basically these are priority order for you. Then it's flywheel engagement  referral stuff to see if there's a way to drive that. Then revenue. How do we make money with  this and how do we make sure we're doing this

52:59

profitably? And only then do you start to  go big on acquisition top-of-funnel growth. Yeah. I may need to do some acquisition stuff  before just to bring enough flow-through, but I'm not obsessing on how scalable is this.  It's just like, yeah, let's get enough people coming through that we can start to take the slack  out. Part of it comes down to that the acquisition

53:20

side is so competitive now that if you're not  really efficient at converting and retaining and monetizing customers, you can't find scalable,  profitable customer acquisition channels. This is fascinating because I think a  lot of people probably do the opposite. Start driving a bunch of growth to a  product, then we'll fix onboarding, then we'll figure out how we're making  money, and referrals comes along there.

53:43

So I think this is really important for people  to hear. So again, the reason you invest first and focus a lot on onboarding/getting people  activated is because that is very correlated to retention and this must-have customer,  this, "I'll be very disappointed," customer. Yeah. And they're at highest risk  of losing them at that point. They-

54:04

They're at highest risk of losing them at that  point. They're probably a little skeptical about a promise that you put out there, but they're  intrigued enough to want to use it. But until you get them to that must-have experience, until  you kind of get them to that aha moment, they're at their high risk of being lost. And so a lot of  people focus on, "Well, I better get their email address or their phone number." But then you're  essentially having to reacquire them at that

54:27

point. So to me, if you can collapse that time  to value, I can give you a couple of incredible examples of when we [inaudible 00:54:38] So at LogMeIn, when we initially tried to grow the business, I was stuck at being able to  spend... I couldn't spend more than $10,000 per

54:47

month profitably trying to grow the business. And  then I dug into the data and I saw that 95% of the people signing up for LogMeIn. So LogMeIn, at the  time free remote access for your computer. And so you install software and you can control it from  any other computer. So 95% of the people signing

55:09

up never once did a remote control session.  And so not surprisingly, then I had to get my monetization off the 5% who did that was really  limiting my ability to find channels that worked. Credit our CEO with this, that I shared the data  with him and he basically told the product team,

55:30

"We are putting a complete freeze on the product  development roadmap." So every single person from product, engineering, design and then also said  to me, "Stop trying to find new channels." The three of us on the marketing side are all going  to focus on improving the signup to usage rate. And so in three months, we improve the signup to  usage rate by a thousand percent. So we went from

55:54

only 5% of people using the product to 50%.  I went back, tried the exact same channels that previously only scaled to $10,000 a month. Now they scaled to a million dollars a month with a three-month payback on marketing dollars  invested. 80% of new users were coming in through word of mouth. So there was this major  inflection point by just focusing on activation.

56:19

This episode is brought to you by Merge.  Product leaders, yes, like you, cringe when they hear the word integration. They're not  fun for you to scope, build, launch or maintain, and integrations probably aren't what led you to  product work in the first place. Lucky for you. The folks at Merge are obsessed with integrations.  Their single API helps SaaS companies launch over

56:41

200 product integrations in weeks, not quarters.  Think of Merge like Plaid, but for everything B2B SaaS. Organizations like Ramp, Dorada and Electric  use Merge to access their customer's accounting data to reconcile bill payments, file storage data  to create searchable databases in their product or HRIS data to auto-provision and de-provision  access for their customer's employees.

57:04

And yes, if you need AI-ready data for your  SaaS product, then Merge is the fastest way to get it. So want to solve your organization's  integration dilemma once and for all? Book and attend a meeting at merge.dev/lenny and  receive a $50 Amazon gift card. That's merge.dev/lenny. What do you find often works  in helping increase activation? I know there's

57:30

a million things that people do, but I guess  what are three or four things that you think people should definitely try to help improve  activation and their onboarding conversion? One of my favorite quotes is a quote from a guy  Kettering, who was a hundred years ago at GM running innovation. And he says, "A problem well  stated is a problem half solved." And so I think

57:53

a lot of it comes down to not the things you try,  but how you deeply understand the problem that's preventing someone from using your product  effectively. And so I'll just give you one example. We had one channel after we made a lot  of these changes and had already driven a ton of improvement in the LogMeIn onboarding. We found  a demand generation channel that was really cheap

58:20

and the economics looked great, but at just  the download step we had a 90% drop off rate. And so we A/B tested a bunch of different things  there to try to improve that conversion rate, and then finally 10 plus tests, not able to improve  it. Finally, someone said, "When these people are

58:41

registering. Why don't we just ask them why they  signed up and didn't download the software?" And so we didn't want to do it in too kind of a creepy  way. So we made it look like a note coming from customer service. This channel was sending 200,000  people a day, so 20,000 people were converting to registering. So we had essentially 20,000 people  we could email and then 18,000 of them who didn't

59:07

download. And so we just asked, "Hey, notice  you haven't had a chance to use the product yet. It looked like it was coming from customer  support. What happened?" And the answer we got back and not a formal survey was, "Oh, this  seemed too good to be true. I didn't believe this was free." I mentioned to you we were one of the  first freemium SaaS products out there. And so

59:29

people were skeptical, especially in a demand  gen channel where they hadn't seen a radio or a TV advertisement from our competitor who was  a premium only product. These were people who were discovering the category for the first time  they were getting there. Once we articulated what the problem was, our next test gave us a 300%  improvement in the download rate, which was...

59:53

We gave them a choice, download a trial of  the paid version or download the free version, put a big graphical check mark next to the free  version. But when they saw we had a business model and a trial of a paid version, the free  version was credible. And so that essentially made that channel work for us. So I think again,  it's that combination of qualitative research,

1:00:14

looking at how others did it. We had this theory,  our previous company had been a game company that didn't require a download. So initially we had  this theory that maybe just downloadable software can't be in the millions of new customers a  month and so we're being unrealistic here. But then we were like, "Are there any  counter examples to that"? And no,

1:00:34

the instant messengers are downloadable and  they have hundreds of millions of customers, so let's study their download and install  process and see if we have any ideas that we could borrow from that. So again, some  inspiration, tried some of those things, it was a combination of just trying a bunch  of different stuff that ultimately led to,

1:00:55

I wouldn't say there was one big  gain, it was a bunch of small gains. Awesome. So a few things for people to try  if they're like, "Hey, how do I improve my activation rate? How do I improve my conversion  rate?" Just drill further into what is stopping people from progressing. Ask them, "Why did you  bounce here? What did you think this was going to be? Why didn't you end up using this?"  Look for inspiration from other products,

1:01:18

I think people probably already know that. You  talked about earlier this idea of the positioning, having a big impact of just figuring out. They want an antivirus software. Let's make that very clear. "Hey we've got the best  antivirus software, that's what we're here for." So there's probably just messaging  that you find works a lot of times, right? I mean your two big levers on driving a  conversion are increase desire, reduce

1:01:41

friction. And so you definitely want to increase  the right desire. Sometimes it can also just be reminding people along the way of what benefits  you're going to get. In the case of LogmeIn, it was probably the most complicated funnel I've  ever seen because you couldn't even get to the aha moment while you're sitting in front of the  computer. You had to actually go to a different

1:02:07

computer and to use the service to remote  control the computer you're in front of. So it's not surprising that there was so many  steps where we could lose people, but we just weren't that intentional about designing each  of those steps initially. And it wasn't until we thought through why would we lose someone at  this step and studying the data, which steps were

1:02:28

we losing the most people at? Then deeply trying  to contextualize why are we losing them there, coming up with a set of tests that we want to run  and then having a good way of deciding which one to test first and ultimately focusing the tests  on the areas where we're losing the most people. The other element of this is coming up  with an activation metric and aligning

1:02:48

on here's what we consider so activated. I  know this is very dependent on the product, but any advice or heuristic for how to help  people decide this is our activated user. I tend to start qualitatively. So just like when  do I think they've had a good enough experience with the product to really know it? And so in  the case of LogMeIn, it was pretty easy. If they

1:03:10

didn't do a remote control session, they didn't  use the product. There was no value along the way there. And then at least try to see if there's a  correlation to long-term retention of doing that. Causation is you need to do some experimentation  to prove causation. At the very least, I want to see that correlation, but if I start with two  or three ideas of what it might be and then

1:03:36

go and study the data, that can help you focus. But again, I don't think there's necessarily one exact right answer of what is that aha moment.  There might be two or three different things. I think it's that intentionality about picking  something that's experience-based and saying, "What is a likely experience that someone's going  to get a good enough taste of this product?" And

1:03:58

then I do see some companies that are like, "Well,  the activation moment should be, they've used it a hundred times." That's going to correlate to  long-term retention, but it's just not very actionable. It's so far down the user experience. So ideally if there's a way that I could get them there in the first session, in the first day,  that's great. And so it's sort of something

1:04:23

that's value that can be experienced super  early. To give you an example from the first company I worked on was a game company, where  I actually flipped it and basically instead of making a traditional funnel where they  could play our games after they signed up, I made our games the advertisements. So basically  we syndicated our games to 40,000 websites.

1:04:48

They started gameplay experience on the other  website, then they would get a message that they now have a qualifying score and if they  register, they'll be in the drawing for the weekly cash prize and then we could pull them into  multiplayer games on the site. And so it's kind of the strategy that YouTube used to grow, but it was  two years before YouTube introduced the approach.

1:05:14

It feels like you basically created Zynga, is what  I'm hearing there. So let's move further down the funnel. So we've talked about activation,  onboarding. The next phase that you focus on is basically some people call this growth  loops, growth engines, flywheels. Basically it's the thing that helps your business grow  and something I am curious if this resonates.

1:05:34

I found there's basically four ways to grow and  usually one of these engines is responsible for almost all of your growth. So what I've seen  is basically you're going to go through sales, you're going to grow through SEO,  you're going to go through virality, word of mouth or paid growth. Does  that resonate? Does that feel right? And I wouldn't say it's necessarily one  or the other. I think Bounce is a really

1:05:59

interesting example where SEO is super important  for Bounce. So people who are essentially saying, "Luggage storage, Paris. Luggage storage..."  Most people when they're trying to find a place to store their luggage, they're starting with  Google, but at the same time, a huge percentage

1:06:20

of the people who use Bounce are dragging their  bag down a street over cobblestones in Paris. And then they pass a sign that says, "Store  your bag here for $5 a day." And it's like, "Oh, no-brainer." And so 10,000 partners around  the world means that there's a lot of people in

1:06:41

the right situation on the demand gen side. One would be, I actually think of kind of... I'm not sure how it would map to this, but  demand generation versus demand harvesting. And so one of those examples would be a...  Demand generation example, when you see the signs when you're passing, it's high context,  right place. And then obviously the demand

1:07:02

harvesting would be anyone who's Googling. And  so they do paid search and organic search there. Interesting. I don't see that  sign approach work often, but I definitely have seen it work. Like  Yelp I think grew in a lot of ways of just little Yelp stickers in all the restaurant.  DoorDash I think probably grows through that. I think every business could  be a little bit different, but for Bounce it makes sense that that  would be a really good opportunity for them.

1:07:28

How do you help a business figure out which  area to bet on? Whether they should go paid, whether they should go SEO, whether they should  hire sales. Sales is probably an easier one. B2B, you're probably going to have to be a sales team, I guess just to help them pick, "Here's  where you have a big opportunity." Again, it kind of comes down as I'm going into  it, I'm thinking what are the realistic customer

1:07:52

acquisition angles for this business? And I want  to have ideally two or three that I'm coming into it with, but it's going to... Obviously  Dropbox is a classic one of like, "Oh man, this product..." User get user is going to be  just a classic. There's file share built into it,

1:08:12

folder collaboration. There's so many  pieces of it that cross from one user to the next. But interestingly, it was  fairly similar to LogMeIn in some senses, it's two businesses are solving similar problems  in different ways. Where LogMeIn, we grew almost

1:08:33

entirely off of paid search. And part of it  again is that for us, we had a competitor that was spending tens of millions of dollars a month  creating the category with a premium only product through radio and TV advertising. GoToMyPC,  they're creating all this latent demand. And

1:08:53

so it just made sense for us to disrupt them with  a freemium service and to insert ourselves in the flow of someone... What was that thing that I  heard about on the TV commercial? And now they go and they Google it and same thing, but free.  So we weren't really pushing for differentiation, but just really trying to harvest that. So I couldn't do that at Dropbox. No one

1:09:18

was looking for when I went there. And so we  tried a little bit with search to see can we make it work on cloud storage or backup or kind  of going to some of these traditional categories. Cloud storage wasn't even a traditional category  at that point, but backup was, and it was fairly expensive and there was just not that much demand  there that way. And so it just made more sense to

1:09:45

focus on the user get user loops at Dropbox. I think basically for each business it's just thinking about what's unique for that business  that is going to open up channel opportunities and everyone's going to be a little bit, I think jaded  from whatever the last thing that worked really

1:10:07

well. They're going to think they can apply it in  the next business. But after enough times myself, I tend to get the most inspiration by just talking  to customers and finding out how did they find it, how do they typically find something like that?  And that starts to give me some ideas as well.

1:10:28

I think that last point is really powerful  and I'm just writing it down. You said, essentially one of your tactics is talking  to users, asking them how did you find this product and how do you normally find products  like this? Is that the second question? I think it's similar to your [inaudible 01:21:02] test.  It's such a simple question, but it's so powerful

1:10:48

because how else will people find your product?  They go to a place to find stuff like this, and I searched Google for folder sharing. There's  so much there that you just skip over. I think the reason that you don't actually  hear people taking the obvious route there a lot of times is because, and I used to be in  the same thing, that people tend to be either

1:11:14

over indexed on qualitative or over indexed  on quantitative. So it's like analytics, I'm going to get all my answers from testing  and analytics or I'm going to get all my answers from traditional customer research. And I  was very much in that initial camp for the first five years of my career. I'm just going  to measure everything and test the heck out of things and find stuff that works. But I had a  VC who was our lead VC at LogMeIn who just said,

1:11:42

"When was the last time you talked to a customer?" Just pushed me to survey and talk to customers all the time. And at first I gave the smart-ass  answer, "I don't care what they say, I care what they do." And he's like, "No, you  got to talk to him." Then just to appease him, I would try to have a conversation every day  because he was in our office a lot and so I could say, "Hey, yeah, I talked to a customer  today," when he would ask me. But I started

1:12:07

finding that my experiments were so much better  the more I talked to customers, and eventually I became very much... The blend of qualitative and  quantitative research leads to much better tests. That is another amazing story and  insight. It's so interesting that people sometimes think of you as growth hacker guy  experiments data, when most of the advice we've

1:12:29

been sharing so far is very qualitative driven,  very survey driven, targeting customers driven. And it is just really hard  to run good experiments when you can't deeply contextualize what's going on. I love this. By the way, I don't know if I knew this. So you helped develop  the Dropbox referral program? I was there at the time. Basically, even when I  first started talking with Drew. Before I came in,

1:12:56

I was like, "I think the way we're going to  grow this business is by leveraging the really passionate customer base and that's what we  need to double down on." And we had tried a similar kind of referral program at Zabni and a  friend who actually started Ring, Jamie Simonoff,

1:13:16

previously had a company called PhoneTag  way before Ring, and he had actually done a lot of the testing on double-sided referral  programs and having incentive on both sides, and he found that that worked the best. And so between what we had tested at Zabni

1:13:37

and those conversations with him, I hadn't  actually seen PayPal yet at that point, what they were doing. But that was kind of like...  It seems like a referral program where we have incentives on both sides is the best way to go.  Interestingly, six months before I was at Dropbox, I was at LogMeIn, and I really thought about  having incentivized referrals at LogMeIn,

1:14:01

but 80% of our new users were coming in through  word of mouth. And I had a hundred million devices connected in on our system, and I was just so  afraid of breaking this growth engine by adding an incentive that I didn't want to risk it. But at Dropbox it was so early. I would still

1:14:21

say no experiment is one person. It happened to  be when I was there, I had some insights that I brought in, but ultimately... The guy who built  it was actually an intern named Albert Knee and he ended up dropping out, I think, out of  MIT to stay with Dropbox for a few years after that. But he was kind of my right-hand  guy to collaborating on growth day to day.

1:14:47

Wow. I would say Dropbox is the referral  program, and the PayPal referral program, as you mentioned, are the two most legendary,  studied, copied referral programs out there. Unfortunately, I think that what they don't  realize is that before the referral program, Dropbox had amazing referral rate. Companies  that are trying to copy it are like,

1:15:08

"Why isn't anyone talking about a product? Let's  add a referral program with incentives." To me, I think it's a great accelerant  when it's already working, but it can't fix it if people don't  want to talk about your product. That's an awesome point and something I  was just going to ask about and just coming back to this topic of growing engagement,  growing referrals as a growth mechanism,

1:15:32

what do you look for to tell you that there's  an opportunity there? And I'll just answer it, partly I've seen exactly what you just  said, which is you need to already have strong word of mouth growth because  referrals kind sits on that and gives you a little more incentive to share. So maybe  do you agree with that, not agree to that? Any other advice on helping figure out is there  some kind of loop here that we can build?

1:15:53

Well, one thing I will say is freemium  when we first started with it, as I said, we were one of the first with it, so it took me a  while to figure out exactly how freemium worked, but to me, freemium towards having a free  and a premium version of your product to really work in any business, it needs to  be that your free product is so good that

1:16:17

people naturally have word of mouth around that  product. And then to be economically viable, you have to have a premium product that's better  enough and differentiated enough that people are going to upgrade to the premium product. But I think a lot of times people are so worried about the second part that they make  the free version not very good and then they're

1:16:39

surprised when word of mouth isn't very strong  there. So I think you have to essentially have two distinct products that are great on  their own. So that would be the one piece, but then obviously companies that have any kind  of collaborative layer to them are going to be

1:17:01

more likely to work well with referral. And  then I think on the engagement side, a lot of it comes down to just the nature of the product. Like Airbnb, you're not going to use it every day unless you're like a vagrant or something and  then you wouldn't have money to pay for it. So

1:17:23

there's kind a natural usage cycle to products  and you want to be able to maximize against that cycle. That's where I was saying, coming back  to the hooked model, I think is a really good way to help to have a framework to think about  how do I improve engagement. One good counter example to that though of the natural frequency  of using a product is Facebook when they change

1:17:51

their North Star Metric from monthly active  users to daily active users. I think, again, just having what gets measured gets managed. Once Facebook was on a daily active user goal, the team suddenly had a lot more incentive  to think about, "How do I bring people back

1:18:12

every day and use this product?" Where when  it was monthly active users, they kind of only got credit for that person for using once  in that month. And even if they used 10 times, they didn't get 10 times the credit. It was  just like a, "Oh, that's cool too." But they weren't sort of measured on that. And so I  think it was sort of a random decision for Mark Zuckerberg to move from a monthly active  to a daily active because they hit 1 billion

1:18:37

monthly active users and they're like, "Okay,  let's go for 1 billion daily active users." But it had a really big impact on making that  product way more addictive to the point where obviously they ended up in Congress or get a lot  of pushback. I'm not sure they went to Congress for that, but they got a lot of pushback for  having a product that's maybe too addictive.

1:18:59

And the same thing carrying into Instagram and  some of the other Meta products or basically anything that is highly engaging. So I do think  the right incentives can actually help a team to focus on it, but there's going to be sort of  a natural usage cycle to any product as well.

1:19:22

I'm glad you mentioned North Star Metrics.  I actually have a post I will link to in the show notes where I collected the  North Star Metrics of 30 different companies to give you some inspiration.  I know this is a deep topic of its own, but just when someone is trying to pick their  own North Star Metrics, which I 1000% agree, informs so much about how your company  operates. It basically focuses everyone's incentives to let's drive this thing. And  that changes so much of what you're building.

1:19:45

Any just bullet point piece of advice for  helping you pick your North Star Metrics? I start with the value that's uncovered through  the [inaudible 01:21:02] test. So with a company, I'll say, "Okay, this is what the must have value  is according to our most passionate customers, and we want to think about a metric that reflects  us delivering that value." And then I'll give them

1:20:11

kind of a framework of ways to think about  a North Star Metric. But I think it's really important for it to be a time capped group  conversation. And if you give a team 30 days, they'll take 30 days. If you give them  six months, they'll take six months. But I think generally a team can come up with a  pretty good North Star Metric after 30 minutes if they have the right raw ingredients and a  checklist of what's important in a North Star

1:20:37

Metric. Something that's not a ratio, it's  something that can be up onto the right over time. So you can keep managing it and feeling  good. It should correlate to revenue growth, but revenue shouldn't be the North Star Metric,  but as you grow value across your customer base,

1:20:58

you should be able to grow revenue  at the same rate. And so there's- Revenue at the same rate. And  so there's some other things, but I think that would be the most important,  is that it's something that could be up and to the right over time and reflects value  that you're delivering to customers. Awesome. And I was going to ask  about revenue in your opinion there. And so your advice is don't  make revenue or North star metric? No. Even Amazon, and again, this is just what I  know of Amazon's as being but monthly purchases,

1:21:25

but someone else might say Amazon, no Amazon's is  GMV or something. But I think monthly purchases is great because it maps to value that people  are getting from Amazon. And so even if I spend say $1000 on a TV set with Amazon versus  $3 on a or $10 on an electric toothbrush,

1:21:52

Amazon from the consumer's perspective  delivered the same value. I needed something, Amazon helped me find that thing. And so units  of value from the customer perspective I think is more important than overall revenue. But  clearly with Amazon focusing on driving more monthly purchases, at least on their store side  of the business, that has helped them become one

1:22:18

of most valuable companies in the world. So I  think focusing on value is, revenue should be a product of doing things. Right. It shouldn't  kind of guide your day-to-day actions. To make this even more concrete for people,  are there some North star metric examples you could share that you've seen that are good? Like  say for Eventbrite or Dropbox or any companies you've worked with? And I'll share one real  quick as you're thinking about it. At Airbnb,

1:22:40

our North star metric was nights booked. And  so it's similar to Amazon. It's not like the money Airbnb made from bookings, but it's like  nights booked and it was really, and basically every experiment ran is like, is this increasing  night booked or is this decreasing night's book? And so that's a really good marketplace  one. Uber obviously weekly rides. I'm

1:23:01

always surprised with the Airbnb, that  there's not a kind of time piece on it, like the weekly rides that you have with  Uber, but maybe it's because it's such an infrequent use case on travel that it  doesn't make sense to focus on. Yeah. Yeah. Why is the timeframe important  to you? Why do you encourage that?

1:23:24

Just daily active users, you saw the  difference between monthly active users and daily active users could change behavior a  lot at Facebook. It gives you a quantifiable way, if you're just kind of taking an aggregate number  over time, it always looks like it's going up. So it's an engagement element  of how often are they engaging. Yeah. But,-

1:23:45

Any others? Any others real quick? Yeah, I mean, I didn't really think about North  star metrics when I was at Dropbox and Eventbrite, like the term itself, but I was thinking about  what is a valuable experience with Dropbox and how do I get people to have that more time?  But I don't even know what they go with today, but maybe files in Dropbox, files access might be  better than just files hosted. And then probably

1:24:13

for Eventbrite, again, I would say weekly tickets  or something like you could say weekly events, but then you have events that don't sell  any tickets where weekly tickets would be more likely to reflect, events are going to  be happy if they're selling tickets and yeah. Okay. Sean, we've gone through so  much stuff. I'm trying to limit

1:24:34

how many more questions we get  through just so that we don't,- We're going long. We're going long, which is amazing. I think  there's so much value here that we're collecting for folks. So let just ask maybe a couple more  quick questions. One is actually from Andrew Chen who is currently partnered at a partner at  A16z. He wrote about growth for the longest time. I think he helped popularize growth hacking  for better or worse with his article and it

1:24:58

being the future of VP of what is it? Growth  Hacking is the new VP of marketing, right, Is the title. So he actually had a question for  you that he shared with me. His question is, growth strategies have changed a  lot over the past decade. What is the biggest difference now versus when  you first started working on growth? When I first started just being data-driven on  customer acquisition was enough to win and being

1:25:22

test and data-driven on customer acquisition.  All the other companies were like CPM focused and so we could do really well just with lots  of testing and some creativity in how it all worked. But that over time as, now I would  say most online marketers are very data and

1:25:47

test drive. They know they need to do lots  of testing. And so to be competitive today, you actually have to be able to be super  efficient at all parts of the business. So again, like how you convert, retain, monetize,  and that's when it gets hard. Getting a marketing

1:26:08

team to be data and test-driven is pretty easy.  Once you start getting into activation and referral and engagement and retention, now you're  talking about the overlap between marketing, product if it's B2B, bringing sales in there,  customer success, and those teams are not used

1:26:29

to working together. And so it's really hard to  drive the collaboration that's needed to have an effective testing program across the entire growth  engine. And that's pretty much any business that's been successful with it, implemented it super  early in the business, and so very few later

1:26:50

stage companies have been able to make much  progress in replicating that type of approach. It's just gotten harder basically.  Things are just getting harder. It's gotten harder, but I think it's possible.  So it's what I obsess about all the time, is how do you get cross-functional teams working together on growth now? And it's still a  huge advantage when you can pull it off.

1:27:13

Okay. Totally unrelated question, going  in completely different direction as we close out our chat. So you came up with ICE,  the very popular way of prioritizing work, which is crazy. I did not know that until I  started prepping for this conversation. What's your thoughts on RICE, the intercom version  of ICE, where R stands for reach, I believe.

1:27:34

Yeah. Thoughts? So I think it's an unnecessary addition, but maybe  I'm just being protective of my original idea, that the I in ICE is impact and it's essentially  saying best case scenario, how much impact could we get from this? And reach is a super important  part of impact. And so I think it's already

1:27:56

factored in the I in ICE. And so I think if  there's anything that I would be accused of, it would be being over simplifying things and I'm  not saying them, but there's a lot of people who approach things with, there's got to be a more  complex way to approach this and that's just not me. And so yeah, more testing is better. No, it doesn't just work like that. I mean,

1:28:23

better tests are better than bad tests, but  just if you have to hold yourself accountable to anything, more testing would be better.  And so I think one just quick note on ICE is that in order to be able to effectively  run a high velocity testing program, you need to be able to source ideas from across  the company. And that's why I came up with ICE,

1:28:46

that if you're having people submit ideas and  you can't tell them why their idea was not chosen, they're just going to get upset and  you're going to waste a lot of time. But if you have a systematic way of being able  to compare ideas, it's more likely that people will be able to get it and they'll  be able to come up with better ideas. I love the way you think, Sean. I have a post on  prioritization where I basically just make the

1:29:10

same argument that there's all these complicated  ways to prioritize. In the end, it's just impact, confidence, and effort and it really works  and rarely is more work. On the other hand, I do also have a guest post called DRICE  by these two guys called Detailed RICE, which actually I think is a really good point  where sometimes it's worth spending like 30

1:29:31

minutes per idea to just really estimate  how long will it take to avoid doing things that are just going to not work and very  unlikely. So we're basically doing this reach piece and spending the time too. Right.  And I think there's a lot of good value there. Yeah. And what I thinks going to be  really interesting is that over time, I think AI is going to actually change our  ability to model out potential outcomes on

1:29:58

experiments and start to, whether it's a more  informed way of doing ICE or replaces ICE, that ultimately probability of outcomes is  something that AI will be pretty good at. Well, amazing segue to the final question.  The actually final question is I wanted to ask you about any ways you've been using AI  or ways you think AI will impact the work

1:30:21

you're doing or other folks are doing? And  maybe you just answered it, but you tell me. No, I'll touch on a couple. One is  that probably the funnest way that I'm using it today. Obviously I've done  it for coming up with experiment ideas, but the funnest way I personally use it is  I get a lot of people asking me for advice,

1:30:42

and I don't have very much time to answer  with thoughtful answers to people. And so almost every question that I get, I go to  ChatGPT say, how would Sean Ellis answer this? And it gives me an initial draft to make  a couple of tweaks and definitely allows me to answer a lot more. So it helps to have a book  that's indexed in there and lots of writing.

1:31:07

That is so funny, and that the question, that's as simple as the prompt  is how would Sean Ellis answer,- Yeah, because then a lot of times it'll  say, Sean Ellis, author of Hacking Growth dah, dah, dah, believes that, and then it'll  obviously pull that part out in the answer. Oh my God. So you're one step away  from a Chrome extension or something that just automatically plugs that into your,- Yeah, exactly. And I can even start  to have my personal assistant maybe

1:31:30

start to answer some of those questions as me, but I'm a little bit afraid to send something  without reviewing it first because,- Absolutely. Sometimes there's stuff that's pretty different  from how I would answer it, but longer term, I actually think, as I said, I think the  cross-functional challenge to growth is a thing that holds a lot of companies back from being  able to implement this a bit later. Mostly product

1:31:53

teams don't want to get direction from marketing  teams. Marketing teams don't want to get direction from product teams, and maybe a growth layer can  help to do these things, but I find that if AI is essentially saying, you're underperforming  in this area of your business, you should drive some experiments in this area. It's a lot harder  to kind of let ego get in the way when it's kind

1:32:19

of dispassionate recommendations from a system. And so I actually think, I think the ability to come up with great experiments is going to keep  growing with AI and identifying opportunities. And then obviously the analytical AI side of things  is going to be really exciting in terms of being,

1:32:39

I do find with most companies, once we get a real  high velocity of experiments going, the bottleneck ends up happening more on the analysis side. And  I think AI will help a lot with that as well. Super cool. These are awesome examples.  Okay, Sean, is there anything else you wanted to share or leave listeners with before  we get to our very exciting lightning round, which we'll go through real fast? Because  we've gone very long and I want to let you go.

1:33:01

Yeah. As I've gone through and done a lot  of workshops and programs with companies, I keep coming back to this advice that I heard  from guy Oleg Yakubenkov, which is it often comes down to asking the right question at the right  time in how you figure things out. And he's a former data scientist from Meta and so where he  basically boils data science down to learning how

1:33:28

to ask the right questions. And so I actually  have a course with him called Gopractice.io, where that's really the big benefit of the  course, is to learn how to ask the right questions and yeah, you learn how to query  them and amplitude, but more importantly, being able to ask the right question. I think  it's kind of cool to hear that from a data scientist from Meta, the importance of that. But every time I'm going through exercises in

1:33:52

my workshops, it almost always comes down  to people who aren't able to come up with the right or a good answer in a business.  It's because they're not asking the obvious question. And as soon as they have, like why  aren't users downloading the software? Let's just ask them that question. That would be  one example from my workshop. Who considers

1:34:17

the product a must have? That part of getting,  to figuring out the must have kind of benefit that then allows you to hone in on product market  fit. And so yeah, right questions, right time I think is a really important way to think about  growth and even getting to product market fit.

1:34:39

I love this advice because I think it gives us a  glimpse into how your brain has developed these really seemingly simple ideas that end up being  really powerful. And it feels like the advice is just think a lot about the question you need to  ask because that'll get you just something that a lot of people just kind of under think or  don't. There's things, maybe it's too simple.

1:35:02

Yeah, or they just jump right into  the solution side of things where they're not really trying to  understand what's going on. Yeah. Yeah. Amazing. Okay, well with that, Sean, we've reached our very exciting  lightning round. Are you ready? I am. All right. Our first question is, what are two or three books you've  recommended most to other people? Increasingly, I'm recommending a book called  Presenting to Win that's been around forever, but

1:35:25

it really helped me with my presenting. And so of  course when I'm out traveling, I'm often sharing the stage with other speakers and yeah, I like to  recommend that one to them. I've already talked about Muriel's Hooked. I recommend that always,  and we'll just stick with two. That's good two.

1:35:47

Within Presenting to Win, is there one tip that sticks with you of here's something  that helped me be a better presenter? Ultimately, confidence in presenting comes  down to having very well organized information that you're going to present. And  when you organize it correctly, you are much more likely to deliver it  with confidence. And so he basically says, if I had a presentation to do and I had an hour to  present, I'd spend 55 minutes creating the right

1:36:12

presentation and then five minutes practicing  it. But yeah, there's a lot more to it, but,- Wow. Amazing. Okay. We'll link to  that book in the show notes. Do you have a favorite recent movie  or TV show you've really enjoyed? Yeah, so I've been binging  the Olympics. I love that, just watching people who worked their ass  off for years and then maybe have 30 seconds

1:36:33

to do the thing that they worked hard for. So  Olympics have been awesome. And then the movie, I actually just saw Blackberry,  I don't know if you've seen that. Oh, the story of the Blackberry? Yeah. I mean obviously we all kind of  know the story, but it was so really, I mean, it's a classic example of  product market fit and then not.

1:36:53

Actually, it's probably even a counter example  to the dangers of the how would you feel if we could no longer use this product? Pretty  sure most people would've said on Blackberry, it's the keyboard, and until iPhone came along,  the keyboard was super important and then suddenly it wasn't. But yeah, it's also interesting  on egos and other things that everybody's

1:37:17

getting friendly in the beginning and then egos  take over and things get a lot harder later on. That was actually a really  good movie. There's also an amazing movie called Tetris. For some  reason, I think of these two together,- Okay. About the story of Tetris, and it's a  similar parallels to those two movies. Awesome. I'll have to see that one. Next question, do you have a favorite product  you've recently discovered that you really love?

1:37:39

I forget the name of it, but I think or it's  called Pack Gear Hanging Suitcase, and I basically like, I've done almost 100,000 miles in travel  this year, and I have another trip scheduled for next week, and I love it because it basically has  all my clothes folded in this little insert that

1:38:00

goes into my suitcase, and then I just pull it out  and hang it up and just makes travel way easier. It's called the Pack Gear Suitcase? Pack Gear Hanging Suitcase Organizer. So cool. Going to check that out.  Two more questions. Do you have a favorite life motto that you often  come back to that you find useful in work or in life? Maybe share  with friends and family sometimes.

1:38:22

Focus on reputation and learning over earnings  has served me super well that, and I'll give you an example. I had two companies when I was  doing a lot of this early interim stuff yeah, 10 plus years ago, and I had two of them where  I talked to the founders afterwards and I could

1:38:43

tell they weren't that stoked on my contributions.  And I offered a full refund to both of them with a thought that like I have this reputation that's,  like I randomly pulled the number and said, my reputation worth $5 million. Why would I possibly  mortgage that reputation for $20,000? And so one

1:39:04

of them, I gave the check back to them and he was  happy to take it, but he had said, "Oh, you can make it up to me. You don't have to give me the  check, just make it up to me by continuing to help me for an unlimited amount of time going forward." I was like, "Oh, take the check." And then the other one said, "No, no, I'm actually really happy  with what you did. We're fine." But the two VCs

1:39:28

who had made those introductions were the first  two to give me term sheets when I went out to raise money for my company. And the pre-money  ultimately ended up being valued at more than double what I had put my personal reputation at.  So I, yeah, I think the, yeah, unfortunately the company didn't do that well itself because of the  elusive product market fit challenges. But yeah,

1:39:50

the learning there of just focus on learning  and reputation. Reputation opened the door to more and more learning. And as I got more  learning, the reputation grew. And so yeah. There's a really good corollary there with  customer support. If someone just hates your product and wants a refund, just give them a  refund and let them move on versus being upset.

1:40:12

Yeah, absolutely. I love that. Final question. You mentioned  to me before we started recording that you were maybe indirectly responsible for  TikTok's success. Maybe share that story. Yeah, I mean, I don't want to overstate it, but  I yeah, my trip around the world that I did three

1:40:32

months ago, I think I wrapped it up. I met with  the original founding growth team at TikTok. They're based in Singapore and they had, I can't  remember what the previous product was called, but they started with the previous  product. And then when TikTok came, they were in place to be the initial growth team  for TikTok, and they basically said all the early

1:40:54

stuff we did to grow TikTok was based on your  writing. So that was before the book came out. So it's a lot of just blogging that I had done,  but it was really, really cool to get that feedback that, yeah, I've always said I have some  really good wins, a lot of unicorns that I helped, but none of the really, really big guys. And  then to hear that, it felt really good to know

1:41:18

that I played some kind of role in TikTok. Of  course, almost the same week they told me that that was Congress having TikTok ban conversations.  So it was good. And at the same time, knowing that maybe if they hadn't read my stuff, Congress  wouldn't be wasting their time on TikTok bans. Oh man. Bittersweet. I hope they don't pull you  into some hearings. Sean, this was incredible.

1:41:42

This was everything I was hoping it'd be. I feel  like we collected so much wisdom here for folks to them figure out product market fit, find product  market fit, iterate, grow their products. So happy we did this. Two final questions. Where can  folks find stuff that you're up to if they want to learn more and maybe work with you in various  ways? And how can listeners be useful to you? Awesome. Yeah, so Seanellis.me is the website  where I kind of link to all the things that I'm

1:42:06

doing. And so that would be one place where, and  there's contact forms on there if anyone wants to reach out. Obviously LinkedIn people can contact  me there. And then I did mention GoPractice. So gopractice.io. Really cool way to learn growth  through a simulated environment of being able to try to grow products. So check out GoPractice  and maybe go to Seanellis.me. When this comes out,

1:42:32

I'll put a special offer on there for  Lenny's listeners so you can save some money. And there's also a LLM AI kind of,- I wasn't directly involved on  that one, but there's yeah, there's some other really cool stuff that  Oleg and the team are doing. Data-driven product management, and the user growth  programs are the ones that I helped with.

1:42:53

Awesome. And then for folks, if they're  wondering, do you do advising? How do you work with companies in case  they're like, hey, I need Sean. Yeah, I mean, so the sweet spot for me on  companies that I go hands-on with are ideally pretty early just after they get to product market  fit and now you know how to measure it. So if you're kind of pre-scale, but you're seeing that  40%, or even if you're a bit earlier than that,

1:43:18

we can start talking earlier. But to me,  that's my favorite time to get in there, build it right from the beginning. It's so  hard to retroactively do these things. And I'll go in for three to six months and I'm all  in full-time, one of the team trying to really help build traction in the business. I do one of  those every maybe year or maybe every year or two

1:43:44

because I purposely burn myself out and then have  fun doing more lecturing and workshops and stuff. Awesome. Well, you might get a  flood of requests after this comes out. Hope you're ready. Sean,  thank you so much for being here. Awesome. Thank you, Lenny. I  really appreciate you having me on. Bye everyone. Thank you so much for listening. If  you found this valuable, you can subscribe to the

1:44:05

show on Apple Podcasts, Spotify, or your favorite  podcast app. Also, please consider giving us a rating or leaving a review as that really helps  other listeners find the podcast. You can find all past episodes or learn more about the show at  LennysPodcasts.com. See you in the next episode.