Lenny's Newsletter · Product & Work
TIER 4 2020-06-02
> ## **Q: I'm struggling to know whether my product has Product-Market Fit. When I talk to my users, do you have any advice on what questions I should ask them to help me know if I’m on the right track?**  Oh, that fabled product-market fit ✨ After I published [this deep dive on product-market fit](https://www.lennyrachitsky.com/p/how-to-know-if-youve-got-productmarket) a couple of months ago, I began getting this excellent follow-up question. And I’ve been looking forward to tackling it ever since. Not being a user researcher, I reached out to my good friend, world-class researcher, and former Airbnb colleague [Matt Gallivan](https://twitter.com/mgallivan?lang=en). Matt has spent over a decade leading research teams at NPR, Facebook, Airbnb, and now at Slack as their Director of Research. He generously agreed to share his wisdom on this topic, which you’ll find below. If you’d like to hear more from Matt, follow him on Twitter at [@mgallivan](https://twitter.com/mgallivan?lang=en). Like any good researcher, I'd answer this by saying “it depends.” And it depends on many factors, but in general, I'd offer the following guidance: ### **If you can easily reach 100+ active users, consider a survey** 📋 Product-market fit happens only when a set of underlying assumptions end up being true. So try to break down what those assumptions might be for your product, and use questions that measure your progress toward confirming them. Ask questions along the lines of: 1. **How would you feel if you could no longer use this product: Very Disappointed, Somewhat Disappointed, Not Disappointed?** *You may have already done some Googling and stumbled upon this question credited to [Sean Ellis](https://blog.growthhackers.com/using-product-market-fit-to-drive-sustainable-growth-58e9124ee8db), and recently popularized by [Rahul Vohra](https://firstround.com/review/how-superhuman-built-an-engine-to-find-product-market-fit/). Ellis claims 40% of users answering “very disappointed” is the critical threshold for his question. This directionally makes sense to me.* 2. **Compared to other ways you do X [where X is some essential function of your product], using this product to do X is generally: Much Better, Somewhat Better, Neither better nor worse, Somewhat Worse, Much Worse?** *What you’ll want to see are non-trivial skews in a positive direction towards your product — even if they’re not majorities.* 3. **The last time you did X, what product or tool did you use to do it, and why? [Leave this one open ended]** *This third question is useful because it's behavioral. It will give you a rough signal of whether or not your product is actually being used at the times you think it should be— something your logs can't see. You’ll want to look for evidence that could suggest people are choosing an alternative to do the job you want your product to do for them. For example, if this question had revealed a plurality of early Airbnb guests saying that they had booked their last vacation on VRBO because they thought it offered more choices for non-urban travel destinations, I’d have told the co-founders that they had a problem on their hands.* Ellis' question is a measure of user affection for your product, the second question is a measure of relative utility, and like I said the third is behavioral. **While product-market fit looks different for every product, it’s hard to imagine a scenario where you’ll attain PMF if people don’t feel some degree of affection for your product, don’t find it useful, and don’t use it 😉** Despite all the easy ways to build and launch a survey these days, they might not always be feasible, so: ### **If you can't easily survey 100+ active users, talk to 6-12 of them** 🗣 Ask questions along the lines of: 1. **Walk me through the last time you did X [where X is some essential function of your product]. For each key decision you made along the way, tell me a bit about why you made those decisions.** *You’ll notice this question doesn’t even refer to your product at all – ask it to understand how people typically go about the key task your product is meant to enable. Then ask the following questions to see how well-suited your product is to replacing or improving that work.* 2. **What are the different products you use to do X? What are the best and worst parts of using those products? The last time you did X using this product, why did you use it instead of something else?** *Again, where X is some essential function of your product.* 3. **Why did you try this product in the first place? What were you hoping it would do for you? How well has it delivered on that hope?** You’ll want to keep an ear out for two things that may signal product-market fit: **a recognition of explicit, superior value relative to competitors or alternatives, and genuine enthusiasm.** There should be clear, qualitative daylight between the ways that people talk about using your product and the ways they use alternatives, and people should have no trouble articulating detailed specifics of why your product is preferable. **There should also be an element of excitement.** If your users are telling you that they *want* to love your product because it has so much potential, then you’re only halfway there. This is where you can rely on your intuition a bit-- it may be tough to define product-market fit, but you’ll know it when you see it. You’ll also know it when you don’t. ### **There are also two questions I'd recommend you*****not*****ask** 🙅♀️ 1. **NPS (Net Promoter Score), or "How likely are you to recommend this product to a friend?** *While touted as a panacea by what a former Airbnb colleague calls "the NPS Industrial Complex," both the underlying science and the practical utility of this question are debatable at best.* 2. **What people want, so that you can go blindly build it** *This is why you'll rarely hear user researchers wasting energy debating the tired old tropes about how Henry Ford or Steve Jobs would never listen to research: it's just blatantly, obviously true that people sometimes don't know what they want until they see it. Asking people what they want isn't research.* So, as I said at the outset, there are many questions you can ask...but there's no magic single question that’ll tell you everything you want to know. Using multiple approaches and triangulating between them is your best bet. **As you collect data, you should be trying to falsify your product's underlying assumptions**: is it true that people are experiencing the problem your product sets out to solve in the first place? What motivated them to try it in the first place? What was the magical way in which they were hoping it might make their life or some task even a little bit better? Understanding people's motivations for using what you've built can sometimes be surprising, but can also lead you down unexpected new paths to achieving a strong product-market fit. Happy researching! ## **Inspiration for the week ahead 🧠** 1. **Understand**: [Trevor Noah perfectly articulating why Black Americans are so frustrated](https://www.youtube.com/watch?v=v4amCfVbA_c&feature=youtu.be) [Watch on YouTube](https://www.youtube.com/watch?v=v4amCfVbA_c) 2. **Plan:** [Killer Mike addressing the protestors: plot, plan, strategize, organize, and mobilize](https://twitter.com/QasimRashid/status/1266602023526506498) 3. **Support**: 20 things you can do right now as an ally  4. **Read**: [75 Things White People Can Do for Racial Justice](https://medium.com/equality-includes-you/what-white-people-can-do-for-racial-justice-f2d18b0e0234) 5. **Donate**: [Cover bail for jailed activists](https://docs.google.com/document/d/1X4-YS3vFn5CLL9QtJSU0xqmTh_h8XilXgOqGAjZISBI/mobilebasic) and [tech leaders matching donations](https://docs.google.com/document/d/11dLmq-zutyMTSbZy_FSXv3b6X2t3NP7jlzvgrAQWqrE/edit) 6. **Listen**: [Listening to Black Voices Amid Murder, Violence, Protest, and Pandemic](https://kottke.org/20/05/listening-to-black-voices-amid-murder-violence-protest-and-pandemic) and [Martin Luther King, Jr. visits Stanford](https://www.youtube.com/watch?time_continue=1474&v=cYK9xGALPrU&feature=emb_logo) [Watch on YouTube](https://www.youtube.com/watch?v=cYK9xGALPrU) > **Riots do not develop out of thin air.** > Certain conditions continue to exist in our society, which must be condemned as rigorously as we condemn riots. > **In the final analysis, a riot is the language of the unheard.** > And what is it that America has failed to hear? > It has failed to hear that the plight of the negro has worsened over the last few years. > It has failed to hear that the promise of freedom and justice have not been met. > It has failed to hear that large segments of white society are more concerned with tranquility and the status quo than justice, equality and humanity. > So, in a real sense, our nation’s summers of riots are caused by our winters of delay. > And, as long as America postpones justice, we stand in the position of having these recurrences of violence and riots over and over again > **Social justice and progress are the absolute guarantors of riot prevention.** That’s it for this week! **If you’re finding this newsletter valuable, consider [sharing it with friends](https://www.lennyrachitsky.com/), or subscribing if you aren’t already.** Sincerely, Lenny 👋