The invisible problems of a million-dollar product
“What the hell is going on?”
It was the winter of 2015, and my analytics manager just showed me the data on our RBT customers. It wasn’t pretty.
From the surface, Brain Trust was doing well: The community was highly engaged, with dozens (or even hundreds) of high-quality comments per day. We’d spent a lot of time creating community norms to encourage great back-and-forth conversations.
Each month, we also released an interview with someone I admired — the people from my “Brain Trust,” including a retired Navy SEAL commander, the world’s best piano saleswoman, a Jeopardy champion, a charisma expert, and many CEOs and authors. People consistently rated these highly.
Plus, students were holding hundreds and hundreds of meetups around the world. Many people were meeting new friends — sometimes best friends, which was a huge thrill for me — through the program.
Yet when we popped open the hood, here’s what we saw:
For the first time ever, our churn — the number of customers leaving the product — was greater than our growth. This was a huge problem.
Imagine your job is to fill up a large bucket with water. You spend thousands of dollars choosing a clean well, testing the water quality, and painstakingly ladling each cup of water… only to discover you’ve got a massive hole in your bucket!
We sat down and did the math. As Andrew Chen points out:
“…successful public SaaS companies try to keep their monthly churn under 2%.”
Our churn rate — or the amount of people who cancel every month — was approximately 5 times higher!
In other words, at our current churn rate, we’d lose nearly 100% of our students — every year.
By the way, this is more common than most people think. However, most businesses either don’t talk about it, or they can’t. Calculating churn requires sophisticated cohort analysis, and many companies don’t have the resources.
I put a dedicated team on it. They looked at the data, made their recommendations, and over the next year, we improved a BUNCH of stuff.
What happens when we invest $100,000 to fix our churn
We doubled down on the production value of the interviews.
We improved the onboarding experience. From their very first day, we made sure each new member felt personally welcomed into the community.
I even personally dropped in to as many meetups as I could.
Feeling hopeful, I revisited the numbers a few months later.
I opened the analysis, holding my breath…
These changes made zero impact on our churn.
This was really frustrating. We were listening to customer feedback, iterating, using “best practices,” and closely tracking our churn. Yet the number wouldn’t budge. None of it made sense to me.
No matter how we sliced the data, we couldn’t figure it out…
…until I got an email from one of our oldest RBT students. She’d been with the program since the beginning, she just canceled, and gave me some honest feedback:
“I just canceled my RBT subscription and wanted to add a thought for your team… I suspect I’m not the only advanced member of RBT to eventually leave because RBT felt small. I say this because I’ve seen many other people who joined RBT in the beginning (and are now doing big things) have since left.”
Wow. This was devastating to hear. Imagine you’d created a program that was a surprising success. You doubled down, grew it…but then you started to notice cracks in armor.
Suddenly, nothing you tried worked. Your best customers eventually left because they outgrew the program. The new customers came in for a while, stuck around a bit, and eventually left, too. Meanwhile, your customers gave you increasingly aggressive feedback because they truly loved the program — but you couldn’t make it work.
In one email, I learned what thousands of data points couldn’t tell me: The oldest, most advanced members of RBT had outgrown the program. It wasn’t valuable to them anymore, so they left. It was the 2017 version of Eternal September.
Our very best customers didn’t care if the interviews looked prettier or if onboarding was any better. (In fact, we found that the biggest value of our program was not the interviews — which cost us tens of thousands of dollars — but the community!)
They needed another level of community and guidance — in fact, they’d been telling us for months! — but we didn’t react in time.
I can understand how frustrating this could be. For example, if you’re new to a city, everything is exciting. Where should I go out tonight? What about that new bar? Have you tried the new dim sum restaurant?
But after a while, you just want to hang out with your friends. And the difference between the older customers and newer customers continued to grow.
We tried a variety of things to keep challenging them, but I also deeply understand that as the business grew, it became tougher and tougher to serve all the different people in the program. We had a mixed bag of beginners and advanced students, all of whom wanted something different — for $49/month.
Meanwhile, as we worked on the rest of the business, we quickly realized we were MASSIVELY growing in every area… except for RBT.
Imagine: one area of your business generates $100 in revenue, while over here, you’re making $10,000 in another area. Where would you invest your time and money?
In fact, what if that $100 program was also costing you $98 to run?
Here’s one snapshot of an analysis we ran:
RBT interviews were totally different. We found fascinating people — people who weren’t on the typical podcast circuit — and did extremely deep research on them. We flew them to NYC, put them up in great hotels, and shot the interviews in our studio. The result was interviews that were totally different than something you can find for free on a random podcast or YouTube show.
As one of our students said:
“So many people have podcasts. They give every guest the same questions, spend maybe 15 mins prepping, then 45 mins recording.
I think people will be shocked to know the actual time it takes to make something truly excellent — and that you aim for that rather than doing something quick and easy.”
In other words, the company spent 150 hours each month on a program that was generating a fraction of our revenues — and many months, we only broke even!
The millions of dollars in revenue were impressive. But it took serious digging to understand the mechanics underneath the numbers.
This is exactly why you see restaurants in your neighborhood packed with a line out the door…and suddenly, one day, they just close.
Once I had this analysis in front of me, it was obvious it wasn’t just our students who outgrew Ramit’s Brain Trust. We, as a company, had outgrown it, too.
In hindsight, it’s easy to say, “Duh Ramit, OF COURSE you should shut it down! Focus on your strengths!” Intellectually, that makes sense.
But data won’t tell you the whole story. Psychologically, it’s not that easy to shut down a program you’ve loved for years and years. More importantly, our students loved that program. It connected thousands of people and gave them a community where they could be their weird selves. People made best friends. They met their boyfriend and girlfriend. For our readers, they felt like they’d truly found their peers.
But it’s the CEO’s job to make the hard decisions.
So in December 2016, I hit delete, and our multimillion dollar product disappeared.
Our greatest growth lies ahead of us
Ramit’s Brain Trust was one of the most rewarding programs we’ve ever created. I used to wake up, grab my phone, and spend my first hour in bed responding to the community. I’ll never forget surprising our students in San Francisco, going bowling in Los Angeles, or closing down that bar in the Philippines.
Most importantly, thousands of people built relationships that continue to this day. Yes, I’m thrilled when I hear students used my material to launch a 6-figure business or land a $25K raise. But meeting someone you connect with, who becomes your best man, your gym buddy, or your partner, that’s rare.
The decision to close RBT down was one of the most difficult ones I’ve had to make. Do I have regrets? Of course. I wish we could have known some of this when we started. I wish we could have put a full team on this to iterate on customer feedback even faster. And I wish we could have cracked the code somehow, in some way.
Knowing what I now know, how would I handle RBT? The truth is, I don’t know. We could have focused more on the community, but a community lacks the “hook” to get new people to join. We could have removed the interviews and doubled down on other areas of RBT, but if there’s one thing a community hates, it’s having something taken away.
In short, would I give it another shot? Yes, definitely. There’s magic here. Our students loved it and our business generated millions of dollars. We just weren’t able to crack the code at the level we want to.
I always remind our team that our greatest growth lies ahead of us. That means to forge ahead, sometimes we have to break something today, so we can rebuild it bigger and better tomorrow.
Going forward, as we build future products for our 1+ million/month readers, we’re listening closer and adapting faster. For example, we listened to our community, who wants to be surrounded by other top performers…in person. So we launched Forefront, our new conference to bring the most ambitious, fun people together, last year.
To all the RBT students who joined us on this journey: Thank you.
Ramit Sethi is a New York Times best-selling author and the CEO of GrowthLab.com, where entrepreneurs go to launch and grow their online businesses. On Tuesday mornings he binge watches “Vanderpump Rules.”