strategy+business is published by PwC Strategy& LLC.
or, sign in with:
strategy and business
 / Spring 2008 / Issue 50(originally published by Booz & Company)


The Myth of Cost-Benefit Analysis

One reason that good data is scarce is that the government office that collects and evaluates regulators’ cost-benefit analyses — the Office of Management and Budget (OMB), whose administrators are appointed by the White House — is also the office that decides what data regulators can gather to support their analyses.

“You’re only allowed to collect the data you need with approval from OMB,” says Roberts. If you need data the office hasn’t approved for collection, “you have to guess” at what the right numbers might be. He recalls a time several years ago when his office director was having tremendous difficulty finding what Roberts terms “realistic data” from industry about the volume of shipments of hazardous materials. “The staff finally came up with 800,000 new shipments a day, with 1.2 million shipments in transit on any given day, which is probably pretty accurate,” said Roberts. “But if someone wanted to see how we got that number, it would be very difficult to support.”

Roberts’s experience underlines another troubling issue: Even if OMB grants an agency the right to request data, cost-benefit studies rely primarily on information provided by private industry — most often, the companies that will be governed by the regulations being formulated. Companies often don’t respond to requests in time, leaving agencies like Roberts’s to extrapolate from small samples. When they do respond, studies show that companies generally overestimate costs and underestimate benefits. Also, most industries insist on data confidentiality, making it impossible to verify the information or hold sources accountable for accuracy.

Once the data is collected, there’s another concern: how it is shaped into the basis for decision making. Today the data is often framed to protect existing industries and technologies and discourage innovation, as demonstrated by two studies of the costs of compliance with a proposed noise standard. The Ruttenberg report noted that in 1974, industry presented to the Occupational Safety and Health Administration (OSHA) an analysis by defense contractor Bolt Beranek and Newman (BBN) that estimated the cost of an 85-decibel noise standard to be $31.6 billion. Another study, released to OSHA by independent industrial engineer and noise expert Glenn Warnaka, estimated the same noise control compliance to cost $11.7 billion.

Why are the two figures so different? The BBN study “ignored new technology being developed in the noise abatement field — in sharp contrast to the Warnaka study, which made newly developing technology a key element in its costs of noise control compliance,” the Ruttenberg report said. BBN’s authors even admitted that they had relied on some of the most expensive procedures available to make their estimates, “whereas Warnaka considered opportunities for redesign or substitution of noisy components of existing equipment.”

Similarly, in the early 1980s, when the National Highway Traffic Safety Administration was considering regulations for fuel economy, U.S. car manufacturers objected, claiming the new rules would be impossibly expensive because the necessary technology did not exist. But foreign car manufacturers, including Volvo, Toyota, and Volkswagen, were already using U.S.-patented products to comply with U.S. fuel economy regulations.

All this dissembling and finagling leads to inconclusive or misleading analyses that serve no one well, including the industries being overseen. This is not a new problem, just one that remains determinedly unaddressed. In 1995, when the Office of Technology Assessment published a retrospective study on the techniques of OSHA analytics, it concluded that “a lack of continuing insights on the potential of leading-edge technology hinders the agency in performing its mission.” And in 1997, in testimony before the House of Representatives, a director in the U.S. General Accounting Office criticized the EPA’s traditional approach to environmental regulation as “precluding innovation.”

Follow Us 
Facebook Twitter LinkedIn Google Plus YouTube RSS strategy+business Digital and Mobile products App Store


Sign up to receive s+b newsletters and get a FREE Strategy eBook

You will initially receive up to two newsletters/week. You can unsubscribe from any newsletter by using the link found in each newsletter.