strategy+business is published by PwC Strategy& Inc.
 
or, sign in with:
strategy and business
Published: May 30, 2006

 
 

Deirdre McCloskey’s Market Path to Virtue

In Defense of “Noisy Data”: Deirdre McCloskey’s Campaign against Significance Testing

Deirdre McCloskey’s faith in pragmatism and empirical research also fuels Size Matters, her unpublished work that attacks a small but essential rivet that holds together much of accepted modern economic research — significance testing. The idea comes from the field of statistics and holds, roughly, that an effect is significant if there is less than a 5 percent likelihood that it occurred by chance. The book, which Professor McCloskey coauthored with her former student Stephen T. Ziliak, now a professor at Roosevelt University in Chicago, explores how economists have come to mistake statistical significance for economic significance, and how that confusion has led to vast amounts of economic research that is irrelevant or can lead to poor economic policy. They argue that instead of relying on the on–off switch of statistical significance, economists should calculate the costs of being wrong, and use those criteria to help set significance levels. Statistical significance testing, say Professors McCloskey and Ziliak, is too mechanical to be used as a decision-making tool; the considered judgment of scientists, which is messier and less formulaic, would ultimately yield better results.

For example, in the early 1980s, the state of Illinois developed a marginal-wage subsidy program designed to reduce the time that workers would need to collect unemployment insurance. The researchers who tested the pilot program determined that the average benefit-to-cost ratio was 4.3 to 1, meaning that Illinois reduced its spending on unemployment insurance by $4.30 for every $1 spent on the subsidy. However, the researchers determined that the finding was not statistically significant because there was too much “noise” around the data: Instead of the 5 percent significance level, the data showed a 12 percent possibility that the reduction in unemployment insurance might have been caused by something other than the marginal-wage subsidy.

“These nitwits said: ‘No. Not worth it! It’s insignificant.’ It’s a perfectly clear example of this confusion between fit and substance,” says Professor McCloskey. “If you’re the governor of the state, you’re a sophisticated person, you don’t ask for guarantees. What you want to know is whether it’s worth doing this program.” It obviously is, says Professor McCloskey, who adds that any reasonable interpretation of the data would conclude that the possibility of a four-to-one benefit-to-cost ratio was too valuable to discard because of a 12 percent risk of being wrong. The correct course of action, contend Professors McCloskey and Ziliak, would have been to continue the wage subsidy while conducting empirical research to determine the other factors that might have influenced a drop in unemployment spending.

The misuse of significance testing extends into other realms, like medicine, with sometimes devastating costs. Epidemiologists blame the test for the abandonment of such treatments as flutamide, which has shown promise for patients with advanced prostate cancer. A controversial National Institutes of Health study on women’s health concluded that eating a low-fat diet does not reduce cancer risk even though subjects who avoided fat clearly showed a 9 percent lower risk of contracting breast cancer; critics of the study point out that the finding was dismissed because the results just missed the threshold of statistical significance — not because the low-fat diet was ineffective. “If more women had been studied or the study had gone on just a little longer, the data very likely would have been statistically meaningful and announced as such,” one biostatistician said in a February 2006 article in the Wall Street Journal.

— A.G.

Reprint No. 06211

Author Profile:


Andrea Gabor (aagabor@aol.com) is the author of several books, including The Capitalist Philosophers: The Geniuses of Modern Business — Their Lives, Times, and Ideas (Three Rivers Press, 2002). She is a professor of business journalism at Baruch College/CUNY.
 
 
 
 
Follow Us 
Facebook Twitter LinkedIn Google Plus YouTube RSS strategy+business Digital and Mobile products App Store

 

Resources

  1. John C. Bogle, The Battle for the Soul of Capitalism (Yale University Press, 2005): Vanguard Fund founder’s acerbic and non-McCloskeyan view of managerial immortality.
  2. Ann Graham, “Lynn Sharpe Paine: The Thought Leader Interview,” s+b, Summer 2003: Research on ethical ambiguity anticipates some of Professor McCloskey’s ideas. Click here.
  3. Arjo Klamer and David Colander, The Making of an Economist (Westview Press, 1990): Provides hard evidence, based on a survey of graduate students, that economists have veered too far into mathematical models and away from real-world problems.
  4. Art Kleiner, “Daniel Yankelovich: The Thought Leader Interview,” s+b, Fall 2005: A different kind of management morality optimist. Click here.
  5. Deirdre N. McCloskey, The Bourgeois Virtues: Ethics for an Age of Commerce (University of Chicago Press, 2006): Tour-de-force defense of capitalism as ethical source.
  6. Deirdre N. McCloskey, If You’re So Smart: The Narrative of Economic Expertise (University of Chicago Press, 1990): Argues that the power of economists rests more on their abilities to persuade than their abilities to predict the future.
  7. Deirdre N. McCloskey, The Rhetoric of Economics (1985; University of Wisconsin Press, 2nd ed., 1998): How economic formalists have taken their reliance on econometrics and mathematical models “absurdly too far.”
  8. Rob Norton, “Derivative Wisdom,” s+b, Spring 2006: Survey of sources of financial wizards who influenced Professor McCloskey. Click here.
  9. Michael Schrage, “Daniel Kahneman: The Thought Leader Interview,” s+b, Winter 2003: The Nobel Prize–winning economist parses the roles of emotion, cognition, and perception in the understanding of business risk. Click here.
  10. Adam Smith, The Theory of Moral Sentiments (1759; Prometheus Books, 2000): Deirdre McCloskey credits the father of modern economics with much of her thinking on the virtues of capitalism.
  11. For more business thought leadership, sign up for s+b’s RSS feeds. Click here.
 
Close
Sign up to receive s+b newsletters and get a FREE Strategy eBook

You will initially receive up to two newsletters/week. You can unsubscribe from any newsletter by using the link found in each newsletter.

Close