Administrative Procedures and Risk Regulation:
Should There Be Peer Review?

Sidney A. Shapiro

John M. Rounds Professor of Law
University of Kansas
School of Law, Lawrence, Kansas 66045

ABSTRACT

OSHA and other agencies currently have extensive procedural obligations to assess the impact of proposed regulations and justify the need for regulation, including mandates to undertake detailed risk assessments. Proposals in Congress would subject these assessments to "independent" and "external" peer review. This reform can be evaluated using three criteria: the likelihood peer review will increase the accuracy of agency decisions; the extent to which peer review will delay such decisions; and the acceptability of peer review to the public, interest groups and the agency. This assessment reveals that peer review has the potential to improve risk assessments, but only if it is used in a manner that recognizes its pitfalls and limitations. Agencies, however, have another, potentially more productive way to improve risk assessment. OSHA should publicly reveal the policy preferences it uses in risk assessment and reveal the extent of the uncertainty that affects its risk estimates.

INTRODUCTION
Statutory Framework

The statutory framework for the Occupational Safety and Health Administration (OSHA) imposes both substantive and procedural obligations. The first obligation defines the substantive standards that OSHA must meet in order to promulgate safety and health regulations. The second obligation specifies what procedures OSHA must use in adopting such regulations.

Debate Over Procedures

The procedures used by OSHA and other agencies to assess safety and health risks have been the subject of considerable debate in the last few years. OSHA's critics assert existing procedures are not sufficient to ground regulations in "sound science." They complain that the agency is prone to distort or misinterpret existing scientific studies. The essence of this criticism is that OSHA tends to err on the side of safety in the assumptions that is plugs into its risk assessment models. To correct this perceived bias, legislation has been introduced to require agency risk studies to undergo "independent" and "external" peer review. Another legislative proposal would block OSHA's regulation of ergonomic injuries until the National Academy of Sciences (NAS) studies the risk of such injuries.

Criteria to Assess Procedures

Professor Cramton advises that any evaluation of administrative procedures, such as peer review, "must rest on a judgment which balances the advantages and disadvantages of each proposal." Cramton proposes that three competing considerations are relevant to assessing the advantages and disadvantages: What is the extent to which a proposed procedure furthers the accurate selection and determination of relevant facts and issues, furthers the efficient "disposition of business" and is acceptable, when viewed in light of statutory objectives, to to the agency, the participants, and the general public? These criteria provide a basis for the following analysis of peer review.

ANALYSIS
Impact on Accuracy

Despite the criticism of agency risk assessments, it is not at all clear that past assessments have been overly conservative. In any case, peer review may not greatly enhance the accuracy of risk assessment for two reasons.

First, peer reviewers, like OSHA assessors, will be hampered by a lack of information. Assessors find themselves in this position because OSHA has a mandate to protect workers before they are injured or become ill. Thus, for some, the complaint that OSHA regulations are not based on "good science" is really a "policy" argument that OSHA should not act until more risk information is available. The issue of when is there sufficient evidence for OSHA to act is an important question, but risk assessors have no special expertise regarding it. For this reason, regulatory beneficiaries are suspicious that efforts to impose peer review are intended to slow the regulatory process rather the promote more accurate decisionmaking.

Second, risk assessment is subject is subject to divergent, socially conditioned interpretations, which makes it difficult to reach consensus concerning the extent of risk posed by a safety or health hazard. In Sheila Jasanoff's terminology, this divergence occurs because "regulatory science" is fundamentally different than "academic science." The latter is governed by established and accepted paradigms and relatively uncontested methodological and quality control standards. By comparison, the standards of assessing quality in "regulatory science" are more fluid, controversial, and sensitive to political factors.

Because risk assessment involves issues that cannot be easily separated into their science and policy components, peer review is most likely to contribute to decisionmaking when committee members encompass a representative range of scientific and philosophical positions. Those who appoint peer reviewers, however, may be tempted to ensure that the majority of a committee will supply the "proper" policy component to risk assessment decisions. This can be relatively easily accomplished if the committee is stacked with scientists whose past actions indicate that they will generally resolve science/policy questions in accordance with the preferences of the person making the appointments.

This potential is one reason why critics favor external peer review, but external committees are not subject to the Federal Advisory Committee Act (FACA), which mandates that the membership of an advisory committee be "fairly balanced in terms of the points of view represented and the functions to be performed by the advisory committee." If Congress passes legislation requiring external peer review, it should impose the same requirement.

Even if an external committee is fairly balanced, it may not be as useful as internal review. As discussed below, peer review works best when it is integrated into an agency's decisionmaking process, because it can create an ongoing dialogue among independent scientists, agency officials, and representatives of interest groups.

Impact on Efficiency

Professor Cramton's second criteria is the extent to which a procedure furthers the efficient disposition of business. The potential delay that peer review might create is of concern because the rulemaking process already suffers from what legal observers have termed "ossification." Agencies are already subject to a number of requirements to analyze the impact of proposed rules which have slowed rulemaking to a crawl. Besides the obligations to analyze imposed by the Administrative Procedure Act (APA), agencies are also subject to the requirements of Executive Order 12866, the Regulatory Flexibility Act and the Unfunded Mandates/Regulatory Reform and Accountability Act.

If peer review reduced OSHA's time and resource burdens, it could decrease ossification, but review is unlikely to have this effect. Groups who oppose the recommendations of the peer reviewers are unlikely to slacken their efforts to have OSHA reject the reviewers' interpretation of the risk data. In addition, external review has the potential to produce an adversarial relationship between outside reviewers and agency staff members, which will prolong the rulemaking process.

Impact on Acceptance

Professor Cramton's final criteria is the extent to which an additional procedure, when viewed in light of the statutory objectives, is acceptable to the agency, the participants, and the general public. As noted, risk assessment inevitably overlaps with issues of risk management. Because risk assessors have no special expertise regarding policy issues, their policy judgments are unlikely to be accepted by rulemaking participants holding contrary views. Likewise, agency officials and staffers holding contrary views will resist the committee's recommendations.

In light of this potential, Sheila Jasanoff recommends that peer review should occur inside an agency and should be structured, "whenever possible, as occasions for multilateral exchange, with opportunities for give-and-take between the experts, the agency, and other interested parties." This proposal recognizes that "claims concerning regulatory science can be made more credible to both lay and expert audiences if the independent scientific community engages with other interests -- including government scientists -- in a process of mutual accommodation." By comparison, when the agency and outside scientists find poised in an adversarial relationship, it is more likely that rifts will develop between each group's interpretation of the data, which damages the credibility of both groups.

Jasanoff's insight is supported by the record of internal scientific advisory committees at the Food and Drug Administration (FDA) and the Environmental Protection Agency (EPA). FDA uses an extensive network of advisory committees which interact with agency staff and outside parties to provide advice to the agency about pending applications for new drug and device approvals. The Science Advisory Board at EPA provides similar functions concerning the agency's rulemaking functions.

CONCLUSIONS

Peer review has the potential to improve agency risk assessments, but only if it is used in a manner that recognizes the pitfalls and limitations of such input. Peer reviewers, like agency assessors, are hampered by limited data and divergent interpretations of that data. At the same time, peer review is likely to prolong an already ossified rulemaking process. Despite these limitations, FDA and EPA have relied on internal scientific advisory committees to good effect. Internal advisory committees can foster an open give and take between the experts, the agency, and other interested parties, a process which can clarify issues and sometimes build a consensus.

Advisory Committees at OSHA

Unfortunately, OSHA is not in a position to form a scientific advisory committee similar to the ones used at FDA and EPA. At OSHA, advisory committees must contain an equal number of representatives from management and labor, and the agency is limited in the number of independent experts it can appoint. Rather than mandate independent peer review, Congress should amend the Occupational Safety and Health Act (OSH) Act to eliminate existing membership requirements and to replace them with a simple requirement that OSHA advisory committees be balanced. In addition, Congress should require OSHA advisory committees to present a complete and full accounting of its deliberations, including an explanation of each potential solution to an issue, which solution the committee recommended, and why that solution was preferable. Along with the requirement that advisory committees be balanced, this obligation will help ensure that OSHA cannot misuse advisory committees to obtain a preordained result.

In the meantime, OSHA can obtain a form of peer review by holding open meetings to discuss its risk assessments. Although FACA prohibits OSHA from inviting the participants jointly to propose recommendations, OSHA can still invite a representative group of independent scientists to critique the agency's risk assessments and discuss their insights with agency staff and representatives of interested parties.

Risk Assessment Disclosures

Besides implementing a form of peer review, OSHA can address the concerns of its critics by making two types of risk assessment disclosures. OSHA should make its policy preferences explicit and reveal the extent of the uncertainty that affects its risk estimates.

As explained earlier, risk assessment is not a value-neutral exercise. This leads OSHA critics to suspect that OSHA has made policy judgments in its risk assessments that are not transparent in public documents. OSHA technical experts should be explicit about the policy preferences that motivated the assumptions they employ in risk assessments. While OSHA's critics undoubtedly would choose different assumptions, they are entitled to insist on an identification and explanation of the policies that drive the agency's risk assessments.

The earlier discussion also revealed that a lack of data limits the accuracy of agency risk assessments. Agencies, however, generally have not done a good job of characterizing the level of uncertainty of agency estimates. Risk estimates are often presented as point estimates unaccompanied by information, such as ranges of risk, which would put the estimates in context. Critics urge agencies to characterize uncertainties and explain in understandable terms the confidence that the public can place in agency predictions. While OSHA and other agencies lack good tools to comply with this request, they should do their best.

i. McGarity, Thomas O. and Sidney A. Shapiro, Workers At Risk: The Failed Promise of the Occupational Safety and Health Administration. Westport, Conn.: Praeger, 1993.

ii. Cramton, Roger: Administrative Procedure Reform: The Effects of S. 1663 on the Conduct of Federal Rate Proceedings. Admin. L. Rev. 16:108 (1964).

iii. McGarity, Thomas O.: Environmental Regulation and the "Cost-Benefit State": A Response To Professor Sunstein. Stanford L. Rev. 49: in press (1997).

iv. Jasanoff, Sheila: The Fifth Branch: Science Advisors As Policy Makers. Cambridge, Mass.; Harvard University Press, 1990.

v. Federal Advisory Committee Act: Title 5 U.S.C. App. 2, Section 5.

vi. McGarity, Thomas O: Some Thoughts On "Deossifying" The Rulemaking Process. Duke L.J. 41:1385 (1992).

vii. Administrative Procedure Act: Title 5 U.S.C. Sections 601, 706.

viii. "Executive Order 12866" Federal Register 58:51735 (30 Sept. 1993).

ix. Regulatory Flexibility Act: Title 5 U.S. Code. Section 601 et seq.

x. Unfunded Mandates Reform Act/Regulatory Accountability and Reform Act: Title 2 U.S. Code. Section 1501 et. seq.

xi. Jasanoff, Sheila: Procedural Choices In Regulatory Science. Risk: Health, Safety, & Environment 4:143 (1993).

xii. Occupational Safety and Health Act: Title 29 U.S. Code, Section 656(b).

xiii. McGarity, Thomas O.: Environmental Regulation and the "Cost-Benefit State": A Response To Professor Sunstein. Stanford L. Rev. 49: in press (1997).

xiv. McGarity, Thomas O. and Sidney A. Shapiro: OSHA's Critics and Regulatory Reform. Wake Forest L. Rev. 31(3):587 (1996).