The Heritage Foundation's critique of EPA: What is science?
Posted March 8, 2011
Last week the Heritage Foundation released a critique of a report by EPA assessing the benefits and costs of the 1990 Clean Air Act amendments (click here for my summary). They should have done a little homework first, perhaps an intro class to research methods. Had they, they might not have set themselves up so badly. Double checking their numbers would have been a good idea too.
A few critical errors made by Heritage undermine its critique:
- Heritage concluded that EPA’s analysis was not credible, for the simple reason that EPA had conducted it. But they neglected a critical protocol followed in good science: EPA’s work was exhaustively peer-reviewed. Its report was evaluated by preeminent academics and institutions (see details below); Heritage’s critique lacked any external examination whatsoever.
- EPA’s analysis was based upon an extensive body of peer-reviewed science (see details below); Heritage’s was based on none. Indeed, it only had one study referenced: a non peer-reviewed report conducted by an industry-funded conservative think tank (see details below).
- Heritage writes: “[I]t is simply preposterous to assume that air quality would worsen unabated over the course of 30 years in the absence of a particular statute…History has proven otherwise…Long before the original CAA was enacted in 1963, industrial emissions were declining as a result of technological advances and efficiency improvements.” What a preposterous argument. Common sense notwithstanding, data show the opposite: pollutants regulated under the CAA were increasing prior to the major CAA statutes, followed by a decisive decline after 1970. (Figure reproduced below).
- Heritage proclaimed the study invalid because it estimated a large range of possible benefits, $250 million at the 5th percentile (loosely speaking, a 5% likelihood) compared to a best estimate of $2 trillion. Two mistakes. First, this reveals an embarrassing understanding of basic statistical analysis. If I tossed a coin 20 times, my best guess on the number of heads would be 10. But it’s also possible only 1 head would come up. Does that make the best guess of 10 invalid? Second, they got the figure wrong: it was $250 billion. Had they gotten it right (or read the abstract, for that matter), they might have realized that even this low end estimate gave a 4/1 benefit cost ratio (see table on page 2 of the abstract, and the first paragraph on page 7-10),
Heritage external peer review: none.
Heritage supporting references:
One non peer-reviewed study by the Mackinac Center for Public Policy, a free market think tank that, according to IRS data, is funded almost exclusively by corporate and conservative foundations (e.g. oil giants ExxonMobil and the Koch brothers). When asked by Detroit’s Metro Times in 1996 on funding sources, the Center’s President Lawrence Reed said: "Our funding sources are primarily foundations … with the rest coming from corporations and individuals," but that "… revealing our contributors would be a tremendous diversion…"
EPA’s peer review council:
- Harvard University; Chair, Dr. James K. Hammitt, Center for Risk Analysis
- Yale School of Forestry and Environmental Studies, Dr. Michelle Bell
- Georgia Institute of Technology, Dr. Armistead (Ted) Russell, Engineering
- Brandeis University, Dr. Linda Bui, Economics
- University of Massachusetts , Dr. Sylvia Brandt, Resource Economics
- University of Maine, Dr. Ivan J. Fernandez, Environmental Sciences
- University of Central Florida, Dr. Shelby Gerking, Economics
- Clark University, Dr. Wayne Gray, Economics
- Boston University School of Public Health, Dr. Jonathan Levy, Environmental Health
- Brigham Young University ,Dr. Arden Pope, Economics
- The National Academies, Dr John Bailar
- Air Pollution Control Division, Department of Environmental Conservation, Vermont Agency of Natural Resources, Mr. Richard L. Poirot, Environmental Analyst
- Resources for the Future Dr. Dallas Burtraw, Economist
- Independent Consultant , Mr. Michael Walsh
- Independent Consultant, Dr. D. Alan Hansen
- National Economic Council, Dr. Nathaniel Keohane
Additional contributors from last year’s peer review council:
- Stanford, Lawrence Goulder, Economics
- Carnegie Melon, Lester B. Lave, Economics
- University of California, Charles Kolstad, Economics
- Harvard University, John Evans, Environmental Science
- Stanford University (Consulting Professor), D. Warner North
- NYU School of Medicine, Morton Lippmann, Environmental medicine
- University of North Carolina-Chapel Hill, Harvey E. Jeffries, Environmental Sciences/Engineering/Public Health
- University of Oregon, Dr. Trudy Ann Cameron, Economics
- University of Maryland, Maureen Cropper, Economics
- College of the Holy Cross, Katherine Kiel, Economics
- University of Maryland, Virginia McConnell, Economics
- Syracuse University, David Popp, Public Administration
- Arizona State University, V. Kerry Smith, Economics
- University of Maryland, Mark Castro, Environmental Science
- University of New Hampshire, Scott Ollinger, Natural Resources,
- Boston University School of Public Health, Aaron Cohen
- North Carolina State University, Christopher Frey, Environmental Engineering
- Clark University George Perkins Marsh Institute (environmental science), Dale Hattis
- University of Maryland, Thomas S. Wallsten, Psychology
- Ministerial Institute, Spain, Nino Kuenzli
- Research Triangle Institute, F. Reed Johnson
- Electric Power Institute, Ronald Wyzga, Scientist and Biostatistician
- Consultant, Lauraine Chestnut
What was external review council’s final assessment of the report? Here’s an excerpt from its 12 page review:
“The Advisory Council on Clean Air Compliance Analysis (Council) has offered technical advice and review to the Agency, including over a dozen advisory reports, during the 10-year period of the EPA effort to evaluate the benefits and costs of the Clean Air Act out to the year 2020…”
“The Council is impressed with the quality, scope, and presentation of the Second Prospective Report. The report provides a state-of-the-art analysis of the benefits and costs of the 1990 CAAA. It is comprehensive in scope, sophisticated in methodology, and is accessible to both specialist and non-specialist readers. The report includes methodological innovations that enhance our understanding of the benefits and costs of air-quality regulations…The Council commends the EPA Project Team for its work.”
EPA supporting evidence:
66 references, including numerous peer-reviewed articles in top journals, research papers from top scholars at preeminent universities, studies from prestigious non-partisan research institutes (e.g. National Academies, Brookings, Research Triangle Park), and analyses by EPA’s external Scientific Advisory Board. These studies themselves drew upon hundreds of peer reviewed analyses.
Pollution levels increasing before 1970s statutes, decreasing thereafter:
The data does not support Heritage’s claim that pollutants regulated under the Clean Air Act were declining on their own before the major 1970 statutes were enacted. The graph shows clearly that pollution levels only began decreasing after being regulated, and were increasing prior to that:
Comments are closed for this post.