I tell this joke all the time in corporate settings, and people laugh hysterically because they know in a mature company that guy’s about to get fired. Learning is a four-letter word in most companies; learning means you failed to do what you said you were going to do, which, in turn, means you’re a bad manager.
At the same time, we love to make fun of the CFOs, accountants, and managers who cancel promising projects right before they’re about to pay off. But you have to look at things from their point of view. If you come back from your great expensive adventure with almost no customers and almost no revenue, it could mean one of two things. You learned something great and you’re on the brink of success, or you’re Bozo the Clown and you’ve accomplished absolutely nothing.
If you can’t tell the difference between an A-plus and an F-minus, that is a total paradigm breakdown. And clearly, we can’t take it for granted that finance can sort out this mess. Traditional accounting metrics—profitability, ROI, net return on assets, IR—all show zero in the early stages, even if you are the next Twitter. That’s not the accountant’s fault or the CFO’s fault. That’s the paradigm’s fault.
There’s no way for the finance people and corporate management to make a good decision, because the metrics they’re trained to look at are the wrong ones. They may end up killing projects on the cusp of greatness, letting others go on with no chance of succeeding, and—almost as bad—pushing the ones that show promise forward too quickly.
For some products, the process of testing and iteration takes a long time. How can you have the patience to make investments over years or decades, when your ROI during that period is guaranteed to be negative? If we measure progress differently, it is possible to sustain that kind of commitment. Lean startup is short-term action in the service of long-term vision. It’s our way of quantifying validated learning, to learn objectively who’s making progress and who’s not.
S+B: What are the units of progress you should monitor?
RIES: We still build spreadsheets—the finance guys insist on it—but they are built off more fundamental assumptions that we can test. This demonstrates that learning something about a customer is worth money to the company.
For example, the first assumption in the spreadsheet might be the number of customers willing to try the minimum viable product every day, beginning with the launch date. That becomes the basis of everything that follows: conversion rate of trial customers to purchasers, their repeat purchase rate, and so on.
If everything goes according to the spreadsheet, five years from now we’ll make $100 million. But suppose that first input is wrong. Say we hypothesized that we could persuade 10 percent of customers to sign up for a free trial. Everything in the spreadsheet is based on that assumption. What if that input is actually 0 percent? The five-year forecast instantly goes from $100 million to zero.
So, the first thing we measure on the path to launching our great new product—our first “learning milestone”—is the accuracy of our hypothesis about customer uptake. We celebrate the successful milestone of discovering what that input is in real life, so that we’re no longer in a spreadsheet fantasy.
Imagine that after you talk to 50 customers in a booth in the store, only one of them takes the product home. In traditional corporate settings, that’s bad news that has to be suppressed. You don’t want anyone to find out that you had a failure, because it means canceling the project. But I’m trying to train the new generation of finance leaders to say, “This is great news. We know where we are. That’s a successful milestone. Check, good job.”