Lenny's Newsletter · Product & Work
TIER 4 2024-12-03
Happy Giving Tuesday! Today is the biggest donation day of the year, and, to celebrate, I’m sharing a unique and inspiring guest post by the team at GiveDirectly, a nonprofit that sends cash directly to people in need. **I’m a huge fan of the work they do, so I’m [also kicking off a campaign through GiveDirectly to raise $100,000](https://www.givedirectly.org/lenny/) to bring a rural village in Kenya out of poverty.** All donations made to this campaign today, on December 3rd, 2024, will be *doubled* thanks to an anonymous donor matching pool (while funds last). [Donate now](https://www.givedirectly.org/lenny/)  Your donations will help all 157 families in a village in Baringo, one of the poorest regions in Kenya. Each family will receive about $850 via digital transfer to spend and invest how they see fit. People living on about $1 a day know what they need, and they spend donations on things like eating more than once per day, sending kids back to school, starting a small business, seeing a doctor, and “simple” stuff like buying their first mattress. For less than half the cost of the average month’s rent in the Bay Area, you can give an entire family a life-changing opportunity. [Donate now](https://www.givedirectly.org/lenny/) Here’s the story of Rose and her family:  > “With three children, no job, and divorced, I take on any manual work I can find to put food on the table. Some days, I wash clothes for other people, other days I gather firewood for restaurants at the local town center, and sometimes I fish to earn a living. When there’s absolutely nothing, I go to the lake, catch some fish, and prepare it for supper with my kids. > > I see the cash transfers from GiveDirectly as a real turning point for me. I plan to open a small retail shop to sell fast-moving goods, giving me a steady income and, hopefully, putting an end to the income uncertainties. I also plan to buy livestock, which I can raise and eventually sell at a profit.” —**Rose in Kenya** [Donate now](https://www.givedirectly.org/lenny/) Here’s the GiveDirectly team on the benefits of a cash-based donation approach—which I love: 1. ***Cash works.** It’s among the most data-driven ways to help ([more than 500 academic studies](https://www.givedirectly.org/research-on-cash-transfers/)—basically A/B tests for charity interventions), and GiveDirectly is building this evidence base by running the [world’s largest UBI study](https://www.givedirectly.org/2023-ubi-results/), among other research.* 1. *It’s proven to work in some ways you might expect—e.g. income, psychological well-being, food security, entrepreneurship, education boosts.* 2. *And some you might not—e.g. improvement in gender equity, reduction in domestic violence, reduction in alcohol and drug use.* 2. ***It’s tech-enabled.** Donations are sent directly to families in extreme poverty via mobile money technology, making it more scalable than other poverty solutions.* 1. *It also lends itself to innovative approaches, e.g. [using AI for targeting](https://www.givedirectly.org/hurricane-relief-2022/) to deliver emergency cash faster or even [before disasters](https://www.givedirectly.org/flood-pilots/).* 3. ***It’s respectful and anti-paternalistic.** Too often, “experts” have said people in poverty need [laptops](https://www.theverge.com/2018/4/16/17233946/olpcs-100-laptop-education-where-is-it-now?_hsenc=p2ANqtz-8xaRn9Gvc8DDjisHc2gr13NE60xo1Xe5aYwrKoXUhAdWD-SRCsNvwXZo0dqNqNhv1_6joF), [water pumps](https://www.theguardian.com/commentisfree/2009/nov/24/africa-charity-water-pumps-roundabouts?_hsenc=p2ANqtz-8xaRn9Gvc8DDjisHc2gr13NE60xo1Xe5aYwrKoXUhAdWD-SRCsNvwXZo0dqNqNhv1_6joF), or [chickens](https://www.vox.com/the-big-idea/2017/3/14/14914996/bill-gates-chickens-cash-africa-poor-development?_hsenc=p2ANqtz-8xaRn9Gvc8DDjisHc2gr13NE60xo1Xe5aYwrKoXUhAdWD-SRCsNvwXZo0dqNqNhv1_6joF)—top-down decisions that haven’t worked. Rather than [“teaching a man to fish,”](https://twitter.com/GiveDirectly/status/1630212646346653696) consider that often people know how to fish but cannot afford the boat. They want to sell cassava, say, or go back to school.* 4. ***It’s simple.** Sometimes people who don’t have money just need money!* My goal is to raise $100,000 and bring this village in Baringo out of poverty. Help me make this dream a reality. [Donate now](https://www.givedirectly.org/lenny/) *All donations are tax-deductible in the United States. GiveDirectly accepts all major credit and debit cards, PayPal, checks, wires, stocks, cryptocurrencies, and more.* Now, on to today’s post. . .  Charities aren’t known for being the most tech-forward. Most nonprofit donation flows look like this:  At [GiveDirectly](https://www.givedirectly.org/), we asked: What would a better online donation experience look like? After only one year of experimenting with this one critical flow, we’ve raised $3 million more in incremental donations *per year*—meaning that about 3,000 more families in extreme poverty receive a life-changing cash transfer every year going forward. In your own product—nonprofit or not—you can apply the same strategy. Here are the experiments that worked for us (and some that didn’t) that you can try to optimize your own funnel: ## **Tiny changes can add up to a big difference** #### **1. 💳 Adding one-click PayPal and mobile wallet checkout options raised an incremental $1.3M per year (a 14% increase in online donations!)** Donors often don’t have their payment information handy, so we wanted to reduce the number of steps to complete a donation by adding an express PayPal option, Apple Pay, and Google Pay at the top of the checkout page. Not only did this make it easier to pay, but we didn’t have to ask for the donor’s name and email address, since we get their info directly from PayPal or the mobile wallet. We also think these brands help serve as a trust signal to donors; 20% of our online donations are now made through Apple or Google Pay.  We learned that adding more payment options gives donors the flexibility to use their preferred method, too. For example, adding ACH direct debit was helpful for donors giving bigger donations, providing an easier alternative to mailing a check *and* helping us save on transaction fees (the ACH fee is capped at $5, while credit card fees are about 2%). We estimate this saves **$50,000** per year in fees alone.  #### **2. 📧 Tweaking the copy asking donors to opt in to emails doubled the subscription rate, driving an incremental $400,000 a year** Email is our primary channel for telling donors about our work and asking them to give, so it’s crucial to maximize the number of donors who subscribe. Historically, only 25% of donors opted in to emails during checkout. After we changed the checkbox copy to be up front about how many emails they should expect to get and why they should subscribe, the share of donors who subscribed more than **doubled** to 55%:  While there’s some selection bias in donors who choose to opt in, those who subscribe retain two to three times as long as donors who don’t. We conservatively estimate that the increase in donor email subscription rate drives **$400,000 a year**. ## **If a test fails initially, try a second version** #### **3. 🗓️ Encouraging monthly donations drove an incremental $500,000 a year** We estimate that the average lifetime value of a recurring GiveDirectly donor is **12 times** that of a one-time donor, so we wanted to test ways to prompt more people to start a monthly donation. ❌ **Didn’t work:** We tried showing a modal asking people to switch to a monthly donation of $X/12 after they started a one-time donation of $X:  This made people seven times as likely to start a monthly donation, but one-time donation conversion decreased by 10%—likely due to the added friction of the extra step. The increase in monthly gifts wasn’t enough to compensate for the decrease in one-time gifts. ✅ **Worked:** We tested simply setting the default donation frequency to be monthly instead of one-time:  One-time donations still decreased slightly, but the lifetime value of the increased monthly donations outweighed the one-time donation decrease in this iteration. Smart defaults FTW! #### **4. 💸 Asking donors to cover transaction fees raised an extra $200,000 a year** We pay around $400,000 in payment transaction fees per year—that could go to life-changing cash transfers for 400 more families in poverty. We wanted to see if donors would be willing to help cover some of these costs. ❌ **Didn’t work:** We added a checkbox for donors to cover the payment processing fees for their donation, with the checkbox **checked** by default:  Surprisingly, the donation conversion rate for the test group with this new checkbox was 6% lower than the control group without it. Our hypothesis: People were put off by seeing a different amount on the donate button than they had originally intended to give. However, 85% of people who completed a donation ended up covering the fees, showing that people were willing to “round up” their donation. ✅ **Worked:** We changed the checkbox to be **unchecked** by default:  The share of donors who covered fees decreased from 85% to 60%, showing the power of defaults—but the conversion decrease disappeared, so we shipped it. When it comes to product nudges and defaults, it’s worth testing a few iterations to see if you can find a win. Staying focused on our North Star metric (total dollars raised) also allowed us to recognize when to accept decreases in other submetrics. ## **Homepage design and interactivity matter** #### **5. 🎨 A homepage with a more polished design and embedded donation form raised an additional $700,000 a year (our second biggest lift ever)** GiveDirectly’s very small fundraising team only had the resources to maintain a pretty bare-bones website for many years. Even until 2023, it was still simple and—dare we say—boring:  We knew it was time to upgrade our homepage, and used it as an opportunity to run a three-arm test.  Our goal was to find out: 1. Does a polished homepage with more recipient voices raise more money than a simple homepage? 2. Does embedding the donation form raise more money than a simple “donate” button? **✅ The result:** A website visitor was **35% more likely to donate** (that’s huge!) if they saw the new design with the embedded donation form instead of the old one. Conversion increased from 1.98% to 2.67% (strong for the nonprofit industry, which averages about 1%). Of this uplift, **20%** was driven by the embedded donation form, and **15%** was driven by the new design. Because of these results, we’ve made other web pages (like [this one about extreme poverty](https://www.givedirectly.org/poverty/)) more robust and are seeing them drive more donations. ## **Lessons from a big bet that failed: matching donors with recipients** We also tested a bigger swing to try to increase donations: trying to match donors with the recipient getting the funds. This is a pretty typical model in the nonprofit world, so we had high hopes. At some point in your life, a charity has probably asked you to “sponsor a child” in Africa. You could browse a “catalog” of children in need, and even filter by categories like gender and age:  This has purportedly been a successful growth model for some charities. But is it actually real? And should we present people as “products” to be selected? Evidence has shown that donors can [feel](https://www.nytimes.com/2016/08/03/world/middleeast/worldvision-palestinians-sponsor-a-child.html) [tricked](https://mothership.sg/2021/02/world-vision-hong-kong-sponsor-a-child/) [when](https://winnipeg.citynews.ca/2022/07/09/money-charity-miseld-world-vision-canada/) they realize their money hasn’t in fact gone to the child they’re “sponsoring.” [Browsing bias](https://ssir.org/articles/entry/behavioral_economics_and_donor_nudges_impulse_or_deliberation) also means that donors may choose who to give to based on attractiveness or skin color. And in charities like Kiva that use a similar model, the featured borrowers [have usually already been paid](https://www.cgdev.org/blog/kiva-not-quite-what-it-seems) by an intermediary[1](#footnote-1). So the sponsorship idea is not even entirely honest. But based on the prevalence of this model, it’s clear that donors want to feel more connected to where their money is going. Could GiveDirectly create a version of donor-recipient matching that was truthful and respectful, and also raise more money for people in poverty? #### **How we tried matching donors and recipients—for real**  We tested it out: Donors could fund a basic income for $40 a month, get matched with the person receiving funds, and receive quarterly emails with photos and quotes about how they were spending the money. How did we try to make our version different from other charities’? 1. **No choosing recipients:** You were matched with the next person in the queue to receive funds, based on our needs-based eligibility criteria. 2. **No [poverty porn](https://www.givedirectly.org/poverty-porn/):** We enrolled only adults, got their informed consent to collect photos and quotes, and didn’t edit their stories. 3. **No middlemen:** GiveDirectly runs our own operations end-to-end, so we knew exactly when cash was delivered and provided real-time updates.  However, there were a lot of operational challenges to make this product a reality: **🤳 Gathering content:** Our teams in the field had to collect stories from recipients every quarter, which involved visiting their villages to gather photos and quotes, translating their stories, and uploading them to our content system. Traveling to remote villages [can be difficult](https://youtu.be/mZKb-UQ5iK8?si=rrNGpdcGDAo0DmPo&t=143), and the team time was costly.  **👩💻 Manual QA:** The Product team had to monitor and troubleshoot any delays in transfers or content collection, make sure photo and quote updates were properly populated, and re-match donors whose recipients relocated. This took more than 20 hours each month. **🙋 High volume of donor inquiries:** The customer service team had to deal with a lot of inbound emails from donors wondering where their email updates were when there were delays, and manually manage people’s donation changes. ❌ **Ultimately, this product didn’t improve conversion, and retention benefits were short-lived** #### **1. There was no significant impact on conversion to donate** Our team ran a two-month A/B homepage test between the donor-recipient matching experience vs. a homepage asking donors to “send *someone* a monthly basic income.” There was no statistically significant difference in the percent of visitors who donated (p = 0.69).  #### **2. Any initial increase in retention eventually faded** The donor-recipient matching experience provided a short-term retention boost, but eventually donors who gave using this product churned at the same rate as other monthly donors.  Ultimately, **we decided to deprecate the product** in late 2023 because the added costs and complexity outweighed any initial retention benefits. #### ✅ **Using the lessons from this experiment, we successfully found a more cost-effective way to connect donors with their impact** We’ve since experimented with other ways to show donors the impact of their gift, but with less complexity. For example, could we link donations to a larger unit (e.g. a village, district, or country) rather than an individual recipient? We found that matching donors to a specific **village** was still very compelling, and was much easier and less costly. We tested an email telling donors their gift went to a specific village in Rwanda vs. an email telling them their gift went to Rwanda in general:  The village email had a **58%** higher click-through rate (huge!) and **more than twice** the rate of positive written responses. And the qualitative feedback was effusive: - “It is a very tangible token that our donation has been of help to people who really need and deserve it. It is an almost emotional experience to read their story and see their photographs.” - “I like seeing where my money is going. And living in the U.S., it’s helpful to be reminded of how others are living.” - “This brought me joy to read. It’s rare that we are able to see any real impact from donations, and I’m glad that I’m a monthly donor. Please keep these coming.” We’ve also applied this approach to our village fundraising campaigns with partners, like what you see today at [givedirectly.org/Lenny](https://givedirectly.org/Lenny).  ## **Our experiment takeaways** We’ve collected many lessons about the specific tests and levers, and also learned a lot about what works for our product experimentation strategy overall. These three main lessons might apply to your product too: **1. Balance small optimizations and big bets:** Running low-effort tests helps balance out the riskiness of a bigger bet, and also helps maintain team morale when experiments don’t go as planned. Getting the low-hanging fruit and basics right can have a bigger payoff than we expect. **2. Get out of the way:** Visitors to our website are already interested in helping families in need, but people have short attention spans. Removing steps in the funnel and providing an effortless checkout experience before a donor changes their mind is one of the most impactful things we can do. **3. Test ideas before building anything:** After our lessons from our donor-recipient matching experiment, we tested matching donors to villages, without engineers needing to write a single line of code. This helped validate that the direction would be valuable to donors before we invested significant resources in building anything. [Donate now](https://www.givedirectly.org/lenny/) *If you donate today, on Giving Tuesday (12/3/24), all donations to the campaign will be doubled from an anonymous donor match pool.* *Have a fulfilling and productive week 🙏* ## Hiring? 👀 I’ve got a white-glove recruiting service specializing in senior product roles (e.g. Directors, VPs, and Heads of Product), where I work with a few select companies to fill their open roles. If you’re hiring, apply to work with us below. [Start hiring](https://www.lennysjobs.com/) **If you’re finding this newsletter valuable, share it with a friend, and consider subscribing if you haven’t already. There are [group discounts](https://www.lennysnewsletter.com/subscribe?group=true), [gift options](https://www.lennysnewsletter.com/subscribe?gift=true), and [referral bonuses](https://www.lennysnewsletter.com/leaderboard) available.** Sincerely, Lenny 👋 [1](#footnote-anchor-1) Kiva’s co-founder and CEO [responded](https://www.cgdev.org/blog/matt-flannery-kiva-ceo-and-co-founder-replies) that pre-disbursal of loans is better because borrowers can get their loans more quickly, and that the model they present to users is simplified for people unfamiliar with microfinance. Kiva ended up making changes to their [website](https://www.kiva.org/about/how) in response to the controversy.