Luckily, this more expansive methodology does exist. It was developed specifically for regulators and lawmakers more than a decade ago. The Committee on Risk Characterization, the Commission on Behavioral and Social Sciences and Education, and the National Research Council joined forces to address the bias, scientific uncertainty, lack of transparency, and data–values dichotomy that derail cost-benefit analysis. Their work culminated in the 1996 report Understanding Risk: Informing Decisions in a Democratic Society.
Its original goal was to find a way to translate the output of a cost-benefit analysis into a document that nontechnical decision makers could understand, but the group quickly expanded its charter. “We asked, How do you find out what the relevant information is — for all the interested and affected parties, not just the [regulatory] agencies?” says Paul Stern, who directed the study. “Quantification is one method, but what are the others?”
In response to this larger question, the committee, chaired by Harvey Fineberg, now president of the Institute of Medicine of the National Academies, described and sanctioned an approach it called the “analytic deliberative process” that elevates the “judgment” side of the equation — call it quantitative logic or human deliberation — to a role in which it is just as important to a defensible analysis as are technical data and calculations. Longer and more complex than a simplistic cost-benefit analysis, the process is in essence a collaborative, multidimensional cost and benefit assessment, developed by an expanded group of participants who are demonstrably interested in, or affected by, the decision to be made.
And it actually succeeds at what the cost-benefit approach has failed to accomplish. Through an iterative process, participants question value judgments and assumptions from a fresh perspective. They challenge one another’s biases and data. They see many dimensions of costs and benefits that were previously invisible to or ignored by specialists. They use values and judgment as a positive force to give context and authority to traditional analysis.
Transparency is the other critical benefit to the process, and one that seems particularly important in light of the nearly irresistible urge to game the analysis that is presented by the cost-benefit approach. “If you have a regulatory system that enshrines a collaborative, analytic, deliberative process like the one we proposed, you’ve created an institutional structure that works directly against outside influence,” says Stern. “It’s designed to figure out the information that the group needs and provide the checks and balances to prove the information is trustworthy, since all the stakeholders have the ongoing opportunity to question each other and resolve disputes.”
One example is the resolution brokered between the citizens of Valdez, Alaska, and the marine oil trade after the 1989 tanker crash and oil spill off the Alaskan coast. The two sides had been engaged in a years-long dispute about what kinds of tug vessels should be deployed in Prince William Sound to help prevent oil spills. Instead of funding the usual competing risk assessments in an effort to influence the decision, the Regional Citizens’ Advisory Council, the oil industry, and the government agencies involved in the decision agreed to jointly sponsor, fund, and support a single deliberative assessment.
A steering committee with representatives from all three groups assembled a research team of technical experts and industry and citizen advisors. Over the course of the proceedings, everyone involved learned about the technical intricacies of maritime risk assessment. The process “increased our understanding of the problem domain,” one member of the research team told a researcher studying the deliberation. “The assumptions were brought out in painful detail and explained.” The team decided that the existing records didn’t provide enough data for a proper risk assessment, so the steering committee helped them find the data they needed. As a result, one of the new tug vessels was deployed in 1997 — with the final risk assessment accepted as authoritative by stakeholders who had formerly been at war.