John Lilly, formerly the CEO and founder of a Web-design firm and corporate incubator called Reactivity, recently recalled what it took to sell an idea to venture capitalists during the dot-com boom of the late 1990s. In a period of 30 weeks, his team generated 30 PowerPoint presentations as “prototypes” for a diverse group of Internet-based startups. Out of these, a combination e-mail and Web browser was chosen as the most promising. Its PowerPoint presentation was fine-tuned and then shown to potential backers. Based primarily on this slide show — there was very little else for the venture capitalists to go on — Reactivity raised more than $100 million for a new company (now defunct) called Zaplet.
“That approach wouldn’t work now,” said Mr. Lilly, currently vice president of business development and operations at Mozilla Corporation. “By and large, venture capitalists only fund Web-based companies that already have proven the ability to attract customer traffic.”
Chalk one up for evidence-based management — the notion that real knowledge in the form of empirical analysis of results is the shortest path to the best business decisions. That may seem obvious, yet few companies follow that precept. Many executives make pivotal strategic choices based on nothing more than business fads or the dubious recommendations of advisors who are afraid to challenge the preconceived judgments of their bosses or the organizational status quo. Quantitative or qualitative data that measures how well the strategy is working is often the last concern. As a result, critical company decision making, relating to acquisitions, restructuring, new product launches, brand marketing, and the like, often takes place in the dark.
A classic case is Hewlett-Packard’s acquisition of Compaq for $25 billion in 2001, a transaction that is, in part, to blame for HP’s less-than-stellar record during the last few years. HP insiders told us that the company had conducted no research on how consumers viewed Compaq products until months after then-CEO Carly Fiorina publicly announced the deal and privately warned her top management team that she didn’t want to hear any dissent pertaining to the acquisition. Ms. Fiorina was dismayed to learn, when it was too late to do anything about it, that consumers considered Compaq products to be subpar, the opposite of what they felt about HP equipment.
If you look at the top-performing companies, there are strong hints that evidence-based management is fundamental to their high ranking. A few examples: Yahoo runs many experiments a day in which its Web site design is subtly varied to see which approach attracts the most visitors and purchases; QVC’s television shopping channel analyzes real-time data to determine the presentation and pitch for specific products; and Enterprise Rent-A-Car, the largest U.S. automobile rental company, sends out more than 100,000 surveys per month to monitor customer service at its outlets.
From our research, we are convinced that when companies base decisions on evidence, they enjoy a competitive advantage. And even when little or no data is available, there are things executives can do that allow them to rely more on evidence and logic and less on guesswork, fear, faith, or hope. For example, qualitative data, such as that gathered on field trips to retail sites for the purpose of testing existing assumptions, can be an extremely powerful form of useful evidence for quick analysis.
By emphasizing the importance of evidence and knowledge, we do not mean to dismiss the value of intuition and innovation. But even hunches, fresh ideas, and inventions should be measured against logical and empirical benchmarks to determine whether they are efficacious ideas or just momentarily exciting thoughts better off abandoned.
Here are two basic guidelines for a successful evidence-based management program:
First, act on the facts. Many managers don’t want to hear bad news, admit that they don’t know something, or learn from others. But with these shortcomings they’re doomed to operate without critical information. An excellent example of a company that sees the value in evidence-based management is DaVita, the kidney dialysis company whose mantra is “no brag, just facts.” Data is so important at DaVita that the company always reports quantitative information on patient outcomes, good or bad, even though it is not required to. And when an important measure, such as the number of patients who don’t show up for treatment, is not routinely collected, it is nonetheless included in company reports with the notation “not available” to encourage employees to figure out how to get the information.
Second, treat the organization as an unfinished prototype. Executives who use evidence-based management best encourage their employees to learn even as they act on what they already know. They regard their companies as a work in progress — one that constantly needs to be tested, probed, and experimented with, to be certain that it is evolving in the right direction. They never view their companies as “not broke, so why fix it?” They are confident enough to act on what is already known (even when knowledge is vague and incomplete), and humble enough to change course, if need be, when new information comes along. As former Intel CEO Andy Grove put it, “None of us has a real understanding of where we are heading. But decisions don’t wait for the picture to be clarified. So you take your shots and clean up the bad ones later.”
The most striking thing about evidence-based management is that it forces executives to be curious, not passive, and to be receptive, not closed, to new information. The difference between an executive who embraces evidence-based management and one who can’t see the value in it is invoked in the joke about the two economists walking down the street who spot a $20 bill on the sidewalk. The first says, “Look, a $20 bill. Let’s pick it up.” The second replies, “It can’t possibly be a $20 bill. If a $20 bill were lying on the sidewalk, someone would have picked it up by now.”
Jeffrey Pfeffer ([email protected]) is professor of organizational behavior at Stanford University Graduate School of Business. He is the coauthor, with Robert Sutton, of Hard Facts, Dangerous Half-Truths, and Total Nonsense: Profiting from Evidence-Based Management (Harvard Business School Press, 2006).
Robert I. Sutton ([email protected]) is professor of management science and engineering at Stanford University. He specializes in organizational creativity and innovation, and the role of wisdom in organizational life and performance.