Skip to contentSkip to navigation

What Are the Measures That Matter?

A 10-year debate between two feuding gurus sheds some light on a vexing business question.

(originally published by Booz & Company)

Like all leading characters in a good feud story, Bob Kaplan and Tom Johnson have become living symbols of something much larger than themselves. Once they were research partners and coauthors and shared their success. But they have not spoken in years, and each has publicly staked his professional reputation on the other one being wrong.

Their quarrel, which has lasted more than 10 years, is at heart a fundamental disagreement about the source of business success. Does it accrue to those who drive their businesses with numerical targets and performance measures, as Professor Kaplan asserts? Or to those who believe, as Professor Johnson argues, that management through measurement is fundamentally dangerous?

The debate, of course, is not just about business measurement. It’s about control. In most companies, top management relies on measurements — not just bottom-line targets, but other numerical goals from “fast-cycle” targets to desired “customer satisfaction” survey results — to signal its priorities. Is that, or is that not, a healthy way to run a company?

To Professor Kaplan, it’s not just healthy, but essential to profitability. Robert S. Kaplan, the Marvin Bower Professor of Leadership Development at Harvard Business School, is the most visible figure behind Activity-Based Costing (also known as ABC) and the Balanced Scorecard (which also is part of the title of the 1996 bestseller The Balanced Scorecard: Translating Strategy into Action, published by Harvard Business School Press, that Professor Kaplan coauthored with consultant David P. Norton). Although ABC and the Balanced Scorecard are derived from accounting methods, Professor Kaplan sees both as full-scale cultural changes for management in general. They break down the implicit cultural barriers between finance and accounting, on the one hand, and operations-oriented management, on the other, all for the sake of developing strategies that encompass both.

Activity-Based Costing, for instance, incorporates into corporate financial calculations the kinds of hidden costs that have traditionally been evident only on the shop floor: errors in a production process as it snowballs out of control, wasted effort in cumbersome part-ordering processes, or time spent traveling from one building to another. Taking advantage of computers to gather this information from assembly-line measurements and employee surveys, ABC divides these costs among particular projects, processes, and products. This means, for instance, if the least profitable 10 percent of products are cut using the ABC method, the cut will be more accurate — increasing profitability more — than it would have been under traditional cost accounting.

Opposing Views
If ABC helps financial controllers see what operations people see, then the other Kaplan method, the Balanced Scorecard, moves in the other direction. It helps managers incorporate into their strategies the insights of accountants — the best accountants, the ones who know how to draw forth from a mass of numerical data those few statistics and results that genuinely matter.

The Scorecard, one version of which was originally developed at Analog Devices Inc. (a semiconductor company based in the Boston area), is a sort of update of the Management by Objectives (MBO) system that Peter Drucker helped pioneer in the 1960s. Under MBO, managers were asked to set financial targets and hold themselves accountable for them. The Balanced Scorecard expanded this to include not just financial targets, but also business process improvement goals, customer satisfaction goals, and “learning and growth” objectives (e.g., “What have you done, this quarter, to improve the capabilities of people in your department?”). The “balance” in the Scorecard is the way it trains managers to consider all four criteria, and evaluates them on all four — thus making it less likely (for instance) that they will release products that meet bottom-line cost targets but that no one wants to buy.

“ABC represents the supply curve from Microeconomics 101,” says Professor Kaplan. “It tells you what things cost, but not what they’re worth. The Balanced Scorecard is like a multidimensional demand curve. It tells you what’s creating value.” Together, he says, the two systems “make the concepts of economics operational for complex organizations.”

That’s where his opponent in the feud draws the line. To H. Thomas Johnson, the Retzlaff Professor of Quality Management at Portland State University in Oregon, the adaptation of microeconomics to management decision making has been a kind of original sin dating back at least to the 1950s. As he explains in his recent book (written with Swedish consultant Anders Bröms), Profit Beyond Measure: Extraordinary Results through Attention to Work and People (Simon & Schuster Inc., Free Press, 2000), economics-dominated business schools mistakenly teach young MBAs to make decisions entirely from quantitative information, rather than from explicit, detailed knowledge of how a company conducts work. “In time, this teaching contributed to the modern obsession in business with ‘looking good’ by the numbers,” writes Professor Johnson, “no matter what damage [it] does to the underlying system of relationships that sustain any human organization.”

Professor Johnson doesn’t like to think of himself as a fervent or proselytizing person, but he comes across as one. Writing about the use of numbers to set priorities and control operations, he uses words like “crippling” and “lethal.” He blames the troubles that mainstream companies get into — for example, the current predicaments of the U.S.’s big three automakers — on the misuse of measurement. He says if companies would focus on the “means” (for instance, designing a production system that makes errors visible and correctable the moment they occur), they wouldn’t have to worry about enforcing targets and goals. Error counts would naturally get lower. The “ends” would take care of themselves.

Even to some of Professor Johnson’s friends, this sounds like a utopian dream sometimes, and he would have an awfully hard time making his case if it weren’t for the fact that one major multinational corporation is successfully running all its factories this way. That corporation is quite possibly the most admired and envied manufacturing organization in the world: the Toyota Motor Corporation.

Dying by the Numbers
Unquestionably, Professor Kaplan is the more successful of the two feuders, at least if you judge by the number of companies adopting his ideas. The Exxon Mobil Corporation’s attractive new retail strategy emerged from a Balanced Scorecard exercise; Fannie Mae, Brown & Root, Cigna, and the city of Charlotte, N.C., are all featured in Professor Kaplan and Mr. Norton’s new book, The Strategy-Focused Organization: How Balanced Scorecard Companies Thrive in the New Business Environment (Harvard Business School Press, 2000). Dozens of companies use ABC, and the apparent value of “stretch targets” and other kinds of performance measures has never been higher.

What, then, does Tom Johnson see that Bob Kaplan does not? Or, more to the point: Which is more likely to succeed? Toyota? Or just about every other well-known manufacturer today?

To get a satisfying answer to that question, you have to look back to 1983, when Professor Kaplan was dean of the Graduate School of Industrial Administration at Carnegie-Mellon University in Pittsburgh. A Westinghouse Electric Company executive named Thomas J. Murrin (now a dean at Duquesne University’s business school) pointed Professor Kaplan to a controversial article in the Harvard Business Review published several years earlier. Called “Managing Our Way to Economic Decline,” by the Harvard Business School’s William J. Abernathy and Robert H. Hayes, the article was the first of a series of broadsides against the tenets of financially oriented management. American companies that lived by the numbers, said the article, were dying by the numbers; they were shutting down profitable product lines because they looked costly on paper, and were making themselves unnecessarily vulnerable to competition from Japan.

Professor Kaplan was a financial guy himself, but he found the argument convincing. When he was asked to speak about this at a major accounting conference, he looked for a business historian to help him trace the roots of the problem. A mutual friend recommended Professor Johnson, who had studied with Harvard’s most eminent management historian, Alfred Chandler. Professors Kaplan and Johnson recognized their symbiotic interests and went on to collaborate on a book for Harvard Business School Press, published in 1987 under the title Relevance Lost: The Rise and Fall of Management Accounting.

Relevance Lost has gone through nine printings since then, enough to make it a business-book classic. I vividly remember my first encounter with it, as a fledgling management historian, looking desperately to understand the influence of financial methods on corporate decision making. Reading Relevance Lost, I felt like I had cracked the code. The historical chapters (written mostly by Professor Johnson) showed how management accounting wasn’t just a feature of the newly emerging large corporations of the 19th century; it probably made them possible. Andrew Carnegie’s watchword, for instance, was “Watch the costs, and the profits will take care of themselves.” Cost analysis gave the Carnegies of American business (and their successors, like General Motors’ Alfred Sloan and General Electric’s Ralph Cordiner) the power to create huge, multifaceted, and yet coherent and consistent enterprises that continually outbudgeted and outmaneuvered their competitors.

But cost accounting per se was no longer enough (argued Professors Kaplan and Johnson) amid global competition, demanding consumers, and cutthroat pressures of the 1970s and later. Indeed, like many remedies that are overused, cost accounting had become poisonously destructive to its hosts. The authors asked rhetorically why it had taken so long for the toxicity of calculations like return on investment to become apparent. They explained it by writing that managers had compensated, below the visible surface, with human judgment. But when short-term pressures increased, and managers spent less and less time in each position, human judgment was diminished. The net effect was to make managers more dependent on the numbers.

To Professor Kaplan fell the task of writing most of the material about current management practice, including two chapters describing potential solutions — since accountants had created this mess, how could they help clean it up? He had recently begun to work with Robin Cooper, a Harvard faculty member whose research focused on innovative cost-management practices, and who was writing a case study of Schrader Bellows, a North Carolina hydraulics components company. The company had connected its MRP data bank (a standard “Manufacturing Resource Planning” computer system for production scheduling, sold by IBM in those days) so as to provide information directly into the assignment of overhead costs to products. The term Activity-Based Costing was not mentioned directly in Relevance Lost, but the prototype ABC practices featured in the book soon became its primary deliverable, and thus the center of both authors’ speaking engagements.

“We didn’t argue,” recalls Professor Johnson. “It was obviously going to be a wave to ride. So we rode it.”

Battle Lines Are Drawn
Then it was Professor Johnson’s turn to be approached by a manufacturing guy. As Professor Johnson recalls, Richard Schoenberger, an industrial engineering professor from the University of Nebraska, pulled him aside after a talk to say, “This is really good stuff. You’ve told the accountants what we industrial engineers have been trying to tell them for decades. But you don’t go far enough. Activity-Based Costing talks about tracing the overhead costs to elements of work. But if you could organize the work differently, the overhead costs wouldn’t be there in the first place. And without those overhead costs, why would you need any cost accounting at all?”

That set Professor Johnson off on his own quest. He began to study Japanese and American quality methods, system dynamics, and management ideas rooted in the “new sciences” of quantum physics and evolutionary biology. (For disclosure’s sake, I should add that this path led him to become a contributor to a book I edited, The Dance of Change: The Challenges to Sustaining Momentum in Learning Organizations, with Peter Senge and others.) By September 1992, Professor Johnson had changed his views enough to publish an article in Management Accounting called “It’s Time to Stop Overselling Activity-Based Concepts.” The result of systems like ABC, he wrote, was “unstable processes, unhappy customers, and loss of jobs.” Professor Kaplan responded only two months later in the same journal, in the form of a Socratic dialogue. “Some supporters,” he wrote, obviously meaning Professor Johnson, “have developed a mystical faith in the ability of [quality improvement] to solve virtually all managerial and organizational problems.”

The battle lines were drawn. The two stopped speaking, and in their next books — Relevance Regained: From Top-Down Control to Bottom-Up Empowerment (Simon & Schuster Inc., Free Press, 1992) by Professor Johnson, and Cost and Effect: Using Integrated Cost Systems to Drive Profitability and Performance (Harvard Business School Press, 1997) by Professor Kaplan and Professor Cooper — they each devoted a chapter to excoriating the ideas of the other.

Soon thereafter, Professor Johnson was invited to study the Toyota system first-hand, particularly in its new plant in Georgetown, Ky. In Profit Beyond Measure, he describes his findings in detail. The plant produces about 500,000 cars per year, employing about 7,500 people to do so. Unlike most automakers, Toyota doesn’t ask its dealers to guess what the most popular packages of options and styles will be and produce its wares accordingly. Instead, it assembles each car to match an individual customer’s specification in real time.

Although Toyota makes some use of quantitative indicators of performance — such as first-pass throughput rates, defect rates, and team leader on-line work rates — they have little to do with operational decision making. Procedures on the shop floor are defined largely by team members and team leaders; everything around them is designed to improve the alertness, interest, and well-being of people working there. The plants are remarkably clean and quiet (as such observers as the auto-industry analyst Maryann Keller have noted). People on the line switch stations every two hours to avoid stress and boredom. A Toyota ergonomics engineer once told Professor Johnson that “coming off a shift should feel like finishing a tough but energizing workout.”

Each station is essentially the supplier of the next station in line (its “internal customer”), providing the components the next station needs at exactly the appropriate moments. This, in turn, means people at each station must be aware of the flow of product through the entire plant. They achieve this awareness because the pace of the assembly line is not set to meet a target based on cost or other financial considerations. It ebbs and flows with the pace of customer demand. (Toyota people call this rhythm “takt time,” after a German word for musical meter that the company borrowed during the 1930s.) Machines and workers almost effortlessly retune themselves with every new product variation. People are attuned to notice inefficiencies — the kinds that might show up weeks later as a number on an Activity-Based Costing spreadsheet — and deal with them immediately.

Cords near every station can be pulled when something “feels” wrong. When a cord is pulled, it does not cause the whole line to shut down (as it probably would in a typical plant, with supervisors fretting about the thousands of dollars lost during the downtime). Instead, support staff members rush to investigate; a part of the line then may halt while activity goes on around it.

Plants like Toyota’s save money in part by giving up the enormous overhead of accounting and control systems. They replace them with trust that, given the appropriate training and technological designs, people will manage production more effectively than numbers ever could. “The problem with managing by data,” Professor Johnson says, “is that it creates a mind-set that leads people to pay less attention to the day-to-day particulars of work.”

Professor Johnson has been criticized for being vague and unconvincing. But the deeper reason for the criticism (like that of W. Edwards Deming before him, who referred to goal setting as “management by fear” and called it “pointless”) is that measurements and rankings seem like the natural way to drive people to improve. Most managers intuitively believe that they can get better results only by setting goals and targets, especially the sophisticated “process drivers” of the Balanced Scorecard and similar methods. If managers, following those targets, cut costs in mechanistic or ineffective ways, then they aren’t disciplined enough. “A cost is not a natural thing to measure, like revenues,” said Professor Kaplan in an interview recently. “It’s a construct; you have to create it.” Without such constructs, he argues, even businesses that emphasize quality can fail financially.

The Amoeba vs. the Crystal
For someone like me, who writes about management without having to be accountable for results, it’s very tempting to side with Professor Johnson and Toyota. But then I think of what David E. Meador said. He is the chief financial officer of DTE Energy Co., and a former financial officer at Chrysler, where he was in charge of implementing an ABC practice. “Some people hear Tom talk and they say, ‘This sounds like taking the company off the deep end. It’s a real distraction from near-term results.’ And I know that frustrates him, because it’s not his intent. But listen, if I don’t drive some near-term results, I’m not going to be in a job. Keep the company competitive and keep me in a job, and then I can go work on some enhancements and refinements.”

In other words, to move your company in the direction of Toyota, you have to give up most of your current practices and your ingrained, habitual belief that things will get done only if they are relentlessly controlled and monitored. Toyota has been refining its manufacturing system for more than 60 years, building on its early experience as a loom manufacturer. By contrast, a viable ABC/Balanced Scorecard system can be created in a year or two.

We know that the benefits of the Johnson approach will be slow to surface, and initial resistance will be great. And we know that the Kaplan approach will catch on quickly, and benefits will surface quickly. But we don’t know the long-term dangers of the Kaplan methods. What if the constant use of “process drivers,” measurements, and stretch goals cripple organizations in the long run, by wearing down their people until they leave or their skills atrophy? This is exactly what Harvard professors Abernathy and Hayes noticed, in the article that started both Professor Johnson and Professor Kaplan on this long intellectual quest.

If Professor Johnson is right, then many of the organizations that embrace ABC and the Balanced Scorecard will exhibit the same kind of decline eventually. Indeed, some early aficionados of ABC now express disillusionment about its results. Robin Cooper recently said, “No one is negating its superior capabilities. Yet, look across all the firms that tried it, and a large number failed to take advantage of the insights it provided.”

To my knowledge, no one has yet conducted the kind of long-term in-depth analysis of various companies’ successes and failures that might help us truly judge which professor is correct. In the meantime, you can be reasonably confident that — other factors being equal — Professor Kaplan’s methods will leave you ahead of the game, able to outperform all competitors in the short run, at least. Except, of course, for those very few companies like Toyota that follow a completely different path to management success. Inevitably, they acquire the reputation of inimitable anomalies, as different from conventional business as an amoeba is from a crystal. The crystal feels like a far surer bet, but only the amoeba is poised to evolve.

Reprint No. 02101


Authors
Art Kleiner,

art@well.com is the “Culture & Change” columnist and a regular contributor of “The Creative Mind” profiles for strategy+business. He teaches at New York University’s Interactive Telecommunications Program. His Web site is www.well.com/user/art. Mr. Kleiner is the author of The Age of Heretics (Doubleday, 1996); his next book, Who Really Matters: The Core Group Theory of Power, Privilege, and Business Success, will be published by Doubleday Currency in August 2003.

 

 

Get s+b's award-winning newsletter delivered to your inbox. Sign up No, thanks
Illustration of flying birds delivering information
Get the newsletter

Sign up now to get our top insights on business strategy and management trends, delivered straight to your inbox twice a week.