The AEI-Brookings Joint Center for Regulatory Studies held a conference on December 15, 1998, to discuss the Environmental Protection Agency's 812 Report. Section 812 of the Clean Air Act Amendments calls for the EPA to produce an analysis of the costs and benefits of the Clean Air Act from 1970 to 1990. After opening remarks from Robert W. Hahn, director of the AEI-Brookings Joint Center for Regulatory Studies, Randall Lutter, a fellow at the Joint Center, presented his critique of the EPA's estimates. Afterward, as members of the Science Advisory Board, which performed a peer review of the report, Maureen L. Cropper from the World Bank and Paul R. Portney from Resources for the Future presented a justification of the EPA=s numbers. Robert E. Litan, codirector of the Joint Center, moderated the discussion and presented closing observations.

Robert W. Hahn, Joint Center

Mr. Hahn provided a brief history of the EPA's 812 Report. "This report aims both to quantify and to monetize the past benefits and costs of clean air programs," Mr. Hahn stated. "[T]he upper bound of these estimates is on the order of $3 trillion, over half of the GDP." Those large benefit estimates raise several questions. First, are those benefits real? Second, if they are real, what are their policy implications? Third, are there aspects of the process that can be improved? Fourth, should the emphasis be on total or marginal benefits? Finally, how should we build on that effort to change or adapt the methodology?

Randall Lutter, Joint Center

Mr. Lutter presented his critique in "An Analysis of the Use of EPA's Clean Air Benefit Estimates in OMB's Draft Report on the Costs and Benefits of Regulation." He stated that although we may argue about the strengths and weaknesses of the report, we should applaud everyone involved with the process because we now understand a lot more about the effects of the Clean Air Act.

The EPA report estimates the economic impact of the direct benefits of the Clean Air Act from 1970 to 1990 to be between $5.6 trillion and $49 trillion dollars in 1990, with a mean of $22 trillion. The direct costs of the Clean Air Act, by comparison, are about $.5 trillion dollars. The mean estimate of the total benefits exceeds the total costs by a factor of about forty-two. Mr. Lutter contended that the report represents a model that will influence forthcoming economic analyses that EPA must prepare. But the analysis underlying the model has several problems. In certain instances the EPA could have improved the analysis. Specifically, equally (if not more) plausible assumptions could have led the EPA to significantly different conclusions.

The first problem Mr. Lutter notes is the valuation of health risk reduction. How does one value large reductions in risks to health? The retrospective study estimates large changes in mortality risk and values them on the basis of estimates of willingness to pay derived from much smaller changes in health risk. The benefits estimates, on an annual basis, turn out to be approximately $13,400 per household or 26 percent of average income. Mr. Lutter contends that the EPA's numbers are high because they are based on studies of small changes in risk reduction that are not pertinent or relevant to much larger changes.

The second problem Mr. Lutter notes within the EPA's study is the selection of a baseline. The baseline selected assumes that no new emissions controls were adopted after 1970. The analysis fails to consider state, local, and private efforts. Had the Clean Air Act not been implemented, that assumption implies that sixty metropolitan areas would have had worse air quality in 1990 than Moscow, six metropolitan areas would have been worse than Bombay, and one would have been worse than Delhi - one of the most polluted cities in the world had the Clean Air Act not been implemented. It would have been more informative to use multiple baselines.

Mr. Lutter's paper raises a host of other issues including overconfidence in the model linking particle pollution and mortality, assumptions regarding the timing of benefits, and the possibility that the agency did not consider important adverse effects.

Despite the review by a "truly distinguished panel" and the extraordinary level of supporting analysis, Mr. Lutter concludes that the benefits estimates put forth by the EPA are implausible and excessively high. Using other, equally plausible assumptions, one reaches significantly smaller estimates. "The EPA analysis does not show you how sensitive its conclusions are to [the agency's] assumptions." Those factors raise questions about the usefulness of the agency-authored benefit-cost analysis, even with extensive peer review and public comment.

Maureen Cropper, World Bank

Ms. Cropper is the chair of the Advisory Council on Clean Air Act Compliance Analysis and spoke "on behalf of the Science Advisory Board (SAB) in this capacity." The role of the SAB was to provide peer review of EPA's benefit-cost analyses. They did review the report, and in their letter to Carol Browner stated, "The Council (SAB) finds that the Retrospective Study Report to Congress by the Agency is a serious, careful study and employs sound methods along with the best available data. While we do not necessarily endorse all details of this study's findings, we believe that as a general matter they are consistent with the weight of available evidence."

Ms. Cropper compared the estimates put forth in Mr. Lutter's comment with those in the EPA report. She examined the mortality benefits associated with particulate matter, which are the "big ticket item." Mr. Lutter estimates the benefits from PM reduction at $1.2 trillion, while the EPA report's mean figure is approximately $16.6 trillion. Despite Randy Lutter's critiques, Ms. Cropper stated that she "stand[s] behind" the EPA report. "What is really at issue here, is the way in which peer reviews are conducted and the criteria that you use to settle on estimates when you don't have measures of what you really want to measure."

Ms. Cropper pointed out that the EPA report represents the best compromises available. She acknowledged the criticisms of Mr. Lutter but claimed that the SAB confronted them. Three main factors lower Mr. Lutter's estimates. He first questions whether a causal relationship exists between particulates and premature mortality. That question leads him to multiply the estimates by .75. Next, he presents the concept of a "lag time" between improving air quality and lowering premature mortality rates. Finally, Mr. Lutter uses (a significantly lower value for statistical life years saved with a figure of $100,000 taken from an article by Garber and Phelps.) Those three factors resulted in a benefit estimate substantially below the one calculated by the EPA.

Mr. Lutter's first two assumptions, regarding lag time and the validation of the model estimating changes in health risk, Ms. Cropper described as medical issues. To resolve that problem, the Council used a subcommittee of medical specialists. They concluded that the weight of the evidence [on the relationship between air pollution and mortality] suggests that you can quantify those benefits. The medical subcommittee also investigated the concept of a fifteen-year lag. The committee concluded that at least one-third of the impact of particulates on mortality occurs immediately. The final problem of valuing risk reductions was hampered by a lack of published information. "There are really no good published studies that ask people sixty-five years of age and over what they would pay for reductions in their risk of dying." The Council chose to rely on the existing literature.

Ms. Cropper concluded her discussion by emphasizing that the EPA 812 Report is valuable because it clearly identifies its assumptions and methodology, thus allowing interested parties to evaluate its merits.

Paul Portney, Resources for the Future

Mr. Portney was also a member of the SAB committee during the peer review process. He states that Mr. Lutter "and others have said that they find it implausible that benefits from the 1970 Clean Air Act amendments are on the order of $1.25 trillion per year. Let's be honest about this. It's preposterous! Does anybody here think that the U.S. public would give up one fifth of GDP to have the benefits from the 1970 Clean Air Act amendments? No one in his right mind can believe that." Having said that, Mr. Portney quickly stated that he Aunderstood perfectly how the EPA got where it got" and that it is not very easy to see how he would have gotten a dramatically different number.

Three factors explain how the EPA arrived at their estimates: the choice of a baseline, the estimates of changes in mortality, and the valuation of those changes. Mr. Portney argued that changes in the baseline would have a large effect on total net benefits but not on the cost-benefit ratio. Regarding the mortality estimate, Mr. Portney noted that although the estimate seemed high to him, "the study was well founded and the best available." Regarding the valuation of extending life, Mr. Portney asserted that the EPA used a standard value for a life saved, estimated at $4.8 million dollars, instead of adjusting for life-years saved. "We all know that we shouldn't value saving three years of life as much as we would value saving thirty or thirty-five years of life." Despite that fact, the impact of that decision is not critical. The change would only lower the benefit-cost ratio from forty or twenty to one to perhaps as low as five to one.

Mr. Portney concluded with a suggestion for economists to address the "real problem" of analyzing costs when considering regulation. The EPA administrator is not allowed to consider costs when setting National Ambient Air Quality Standards under the Clean Air Act. Until both the costs and benefits can be considered explicitly, the impact of benefit-cost analysis on policy will be limited.

Robert E. Litan, Joint Center

Mr. Litan concluded the conference by characterizing Mr. Lutter's presentation as a statement "that the benefits are certainly not as large as the EPA thinks they are, but the Clean Air Act was still worth it." He then posed a number of key questions raised at the conference. For instance, what gives benefit-cost analysis a bad name? Does the huge range of benefits add uncertainty? Does the implausibility of the large numbers make people suspicious? Despite the range and size of the numbers, how else can one make informed decisions on regulation? Benefit-cost analysis is certainly not the only criterion involved in regulatory decisionmaking, but it should be an important feature. Studies such as the EPA's report on the costs and benefits of the Clean Air Act and Mr. Lutter's comments on that report will help determine the role of benefit-cost analysis in the regulatory process.