COMMENTS OF JAMES D.

WERNER SENIOR ENVIRONMENTAL ENGINEER ON BEHALF OF THE NATURAL RESOURCES DEFENSE COUNCIL ON THE Carnegie Commission Report (October 1992) -"Facing Toward Government: Nongovernmental Organizations and Government's Quest for Scientific and Technical Guidance"

FEBRUARY 16, 1993

INTRODUCTION Perhaps not since the time of Sir Francis Bacon has the challenge of better integrating science and technology into public policy been so salient as today. The Carnegie Commission on Science, Technology, and Government has addressed this vital issue in a series of reports1; the latest, "Facing Toward Government...", reviews a peculiarly American aspect of it - the role of science and technology (S&T) nongovernmental organizations (NGOs). The Task Force asserts that, "it is desirable for more scientists and engineers...to become involved in [S&T] policy activities and public affairs," and that the role of the Task Force was to "provide[] some of the principles that must underlie increased engagement,"2 Unfortunately, although the report may arguably acheive this modest goal, it is not rigorous enough to recommend as a useful guide either to NGOs wishing to become more involved in public policy or S&T policy veterans seeking to improve the quality of their input. In addition, the report seems curiously fixated on the National Academy of Sciences as a modern day "House of Solomon,"3 seemingly unable to draw examples from other arenas.4

Before addressing the content the report, a couple of structural issues warrant comment, beginning with the composition of the Task Force. Despite the impressive credentials of the Task Force members, the biographies do not indicate that any of the Task Force members have any experience working as advocacy NGOs, according to the report's own typology.5 Because advocacy NGOs normally have more regular interaction with government and policy makers than other types of S&T NGOs, inclusion of representatives from advocacy NGOs would have added much insight and some balance to the Task Force. Moreover, the absence may help explain some of the substantive omissions of the report. Also, the report fails to note that it was written in the context of a series of Carnegie Commission reports on related topics that, together comprise a broader examination of science, technology and government. These other reports, which deal with topics such as S&T support for states and planning issues could have helped support certain points in this review of NGO issues. Finally, the report is sometimes written is an obscure style that gives scientists and engineers a bad name. For example, the report seems to take a swing at Congressman John Dingell's investigation of university overhead charges, with an oblique reference.6 Similarly, the report suggests that certain NGOs, such as National Laboratories7 were spared the budget knife in the early 1980s because they were viewed as sympathetic to the political views of the day (e.g., support for Star Wars and nuclear testing). In addition to dense prose, the report fails to follow up with the logical conclusion that this Stalin-like pattern of "defunding NGOs suspected of unsympathetic policy tendencies" should be corrected.8 Despite these shortcomings, the report provides some useful contributions in this important field - it poses a good set of questions9 to be answered by an NGO getting involved in public policy and a useful, albeit incomplete, set of recommendations.10

1.

The Report Fails to Address the Proper Role of S&T NGOs Where Insufficient Scientific Data Exist.

How should NGOs and government act when faced with uncertainty in pertinent scientific data? This issue is particularly important because situations where some dispute remains about the scientific basis for a policy decision are probably more numerous than where consensus exists. Examples abound: What data should be considered necessary and sufficient for judging the health effects of exposure to the millions of chemicals in public commerce? How much evidence is necessary to support taking action on global warming? What level of response is required? What degree of certainty is required for assuring the safety of plans to deal with nuclear waste - some of which remain hazardous for more than 10,000 years?

In many cases, S&T NGOs urge waiting until more evidence is available before rendering a judgement about risk (e.g., chemicals currently on the market and global warming). In other cases, S&T organizations recommend making a decision on the basis of limited data (e.g., drugs and chemicals awaiting market approval, and nuclear waste disposal facilities). Good public policy often demands that decisions be made in the face of scientific uncertainty. This common dilemma is neither a failure of scientists nor policy makers, but simply a fundamental difference in how each field operates. The greatest contribution of S&T NGOs is not to skew available evidence to support a policy preference as "scientifically and technically rational", but instead to provide a realistic appraisal of the current state of knowledge, of the potential for resolving the uncertainties (including costs and time), and of the potential implications for inaction. It is then up to the policy-makers to render a decision based on available information.11 At a minimum, S&T NGOs should explicitly acknowledge and respect this difference between science and policy making.12 To its credit, the report warns: "NGOs should not go lightly into policy research or policy advocacy." Scientific uncertainty is a predictable problem in this minefield for which the Task Force might have usefully provided some guidance.

2.

The Report Fails to Address Information Access Problems for NGOs Attempting to Provide Advice to Government.

For individuals at NGOs accustomed to working in a collegial setting among academics, the information restrictions often imposed by the government may be a rude awakening. In policy battles where information is power and data are the bullets, the government is often reluctant to arm its opponents. Limitations on data access are particularly frustrating when government agencies use allegedly "objective scientific facts" as a smokescreen to promote subjective policy positions. Several methods are used to stymie independent review of public policy decisions using facts or analyses gleaned from the government. First, the most common method is to follow the rule: information delayed is information denied. Many documents must be obtained through Freedom of Information Act requests, which commonly take years to fill - far too long for use in formulating advice on annual budgets. In some cases, government agencies will mail a requested report the day after a key Congressional vote or the close of a comment period on a regulation. Health data that could be used for independent epidemiological studies has also been withheld.13 Second, information is often released only in a form that makes analysis very difficult.14 Third, the most pernicious method of withholding information is to restrict it as a national security secret or another of a variety of classification techniques.15 The problem of information access is a regular stumbling block for those who would advise government on S&T policy issues, and deserves some discussion in the report. Moreover, the report's recommendations section could have usefully called for collective action by NGOs to eliminate unreasonable restrictions on access to government information.

3.

The Report Fails to Provide Much-Needed Guidance on Avoiding Bias and Conflicts of Interest.

Perhaps because of the composition of the Task Force, the report gives short shrift to the critical problem of avoiding biases an conflicts of interest in advising public policy decision makers. To be fair, the report mentions the issue, in a section on "Quality": [i]t is all to[o] easy to begin to sacrifice objectivity for a comfortable relationship with a sponsor.16 However, because this report presumes to lead S&T NGOs into the fray of government policymaking, this crucial topic deserved a much more rigorous treatment than it received. The report would have been more effective and even-handed if it had examined all potential sources of individual and organizational bias among NGOs. Instead, the report unfairly singles out "advocacy" NGOs17 as being more susceptable to biases, using a worn and inaccurate stereotype: [A]dvocacy organizations may have incentives to exaggerate and sensationalize scientific findings to attract members and funds.18 Completely missing is any broader review of the sources of bias in other types of NGOs or ways of dealing with them. Anyone who has attended even a single scientific meeting, from a brownbag University seminar to a national conference, has probably witnessed scientists overstating the value and validity of their findings.19 Even the august National Academy of Sciences, which soars, according to the Task Force, to "Olympian heights"20 is vulnerable to this temptation of spinning its results to the advantage of its client agencies. For example, last year, in the midst of a Congressional debate on legislation that would help the Energy Department open its Waste Isolation Pilot Plant, a NAS panel released a review critical of the Department's test plan. Predictably, few Congressional staff and fewer Members read the report compared to those who read the newspaper headlines about it, which were thought to be inaccurate and damaging to DOE's prospects for passage of the legislation. In response to a request from DOE, the President of the NAS, Frank Press, quickly sent a one-page "clarification" letter, which appeared to soften the criticism of the original report, to the three Committee Chairmen in the U.S. House of Representatives with primary jurisdiction over the legislation being considered.21 Scientists involved with the technical report believed that the letter was not an accurate summary of the original report, but appeared to lend support to DOE's plans from the NAS at an extremely critical juncture.22 Unfortunately, because of the rush to release it to the news media and Congressmen involved in the intense negotiations at the time, Press' letter was not reviewed by individuals familiar with the report, contrary to the normally rigorous NAS review process. Anecdotes such as this cannot reveal whether one type of NGO is more susceptable to bias than another. Such examples can suggest, however, some of the subtle ways that all S&T NGOs may,

intentionally or unintentionally, influence policy debates.23 Moreover, because of the NAS's established procedures for avoiding bias, this example helps illustrate how difficult it is to avoid the appearance of a lack of objectivity. The basic point is that NGOs may influence public policy outside the four corners of their reports, regardless of how objective and credible they may be, as a result of factors such as timing (releasing findings prior to a key committee vote) and context (holding press conferences to facilitate easy media access or quietly mailing out reports). The Task Force could have helped identify these pitfalls. Having acknowledged that it is virtually impossible to find experts without some bias, and analyzed the potential mechanisms for bias to creep into S&T advice to government, the report could have provided some practical assistance to S&T NGOs new to the policy arena in avoiding even the appearance of bias.24 The report's suggestion that the NAS set down its procedures in writing to help others attempting to avoid bias is a useful step in that direction.25 Other ideas that might have made a useful contribution include: Explicitly revealing any potential sources of bias or conflicts of interest (e.g., previous, current or prospective contracts; personal relationships, etc.). This should be expressed before deliberations of review panels and be recorded in individuals' biographies, instead of the regular practice of selectively extracting credentials. Members of EPA's Science Advisory Board have recently been called upon to disclose organizational and contractual affiliations. Balancing the composition of review panels so that they are as representative as possible of the range of views in the field. Employing an ombudsperson to ferret out possible sources of bias and to remind members of disclosure responsibilities. Ironically, by excluding representatives of advocacy NGOs from the Task Force, the report missed an opportunity to receive input from the groups who may have the most experience at identifying biases and obtaining recommendations for avoiding bias.

4.

The Report Failed to Outline Specific Suggestions for Follow-up.

Two particular opportunities for followup could have been included as part of the Task Force report. First, and perhaps most importantly, the report fails to outline an implementation plan and thereby fails to heed its own admonition against letting useful analysis languish. Implementation could have included: follow-up reviews of specific issues (e.g., eliminating unnecessary restrictions on government information, and identifying and eliminating bias); distributing the report systematically to S&T NGOs; arranging to have the findings presented at major S&T NGO meetings and conferences (e.g., AAAS, ACS, etc.); and organizing training sessions for S&T NGOs interested in expanding their abilities

to interact with government policy makers. Second, recognizing the unique role of S&T NGOs in the U.S., the Task Force could have encouraged efforts through existing or new NGOs to export the idea of forming or enhancing the effectiveness of NGOs in other countries. In Russia and Eastern Europe, the concept of an NGO is often so foreign, it is difficult to explain. Yet once the translation succeeds, tremendous enthusiasm usually follows. NGOs are an American export worth promoting.

February 16, 1993

Mr. Evan R. Ferguson Director of Programs Sigma Xi 99 Alexander Drive P.O. Box 13975 Research Triangle Park, NC 27709 RE: Critique of Carnegie Commission Report - "Facing Toward Government: Nongovernmental Organizations and Government's Quest for Scientific and Technical Guidance"

Dear Mr. Ferguson:

I am pleased to enclose our response to your December 4, 1993 letter requesting comments on the above-captioned report. Thank you for the opportunity to review this report on a matter of vital importance.

We hope that this critique is helpful and look forward to working cooperatively again in the future. Please do not hesitate to call if you have any questions.

Sincerely,

James D. Werner Senior Environmental Engineer

cc:

Frances Beinecke, NRDC/NYO

XXX The report also fails to acknowledge the range of methods advocacy NGOs use in their work, stereotypically referring solely to their "extensive reli[ance] on litigation to achieve their ends."26 This characterization results in a perpetuation of an unbalanced perspective of advocacy NGOs without any mention of other aspects of their work that actually occupies more time such as research and writing for highly-regarded technical books, reports, and articles in peer-reviewed journals.27 The awards conferred on advocacy NGOs illustrate further how blurry the line is between advocacy and other S&T NGOs.28

NOTES

Other reports include "Science and Technology in U.S. International Affairs" (January 1992), "Science, Technology and Congress: Expert Advice and the Decision-Making Process" (February 1991), "Science, Technology and States in America's Third Century". (September 1992), and "Enabling the Future: Linking Science and Technology to Societal Goals" (September 1992).
2

"Facing Government...", p. 64.

Sir Francis Bacon envisioned in New Atlantis (1624) a House of Solomon where wise men would rule an inductively reasoned and ordered society.
4

The focus on the NAS may result from the experience of the project director who served there for more than ten years.
5

However, Mr. Drayton clearly offers considerable expertise because of his experience dealing with advocacy NGOs as an EPA official and funding NGOs in his current position. Given the size of the Task Force (17 + one staffer), and the imbalance toward other types of NGOs (4 honorific members, 1 consultative, 2 federative, 2 discipline or professional, 7 policy, and 1 "quasi-NGO", using the Task Force's typology) some diversification was probably possible by shifting the balance of members among the various types of NGOs.
6

"roundabout hostility by political forces in the guise of administrative propriety or accounting correctness [of] research universities in relation to indirect costs." "Facing Government...", p. 53.

These are included in the report as "quasi NGOs", or "QUANGOs", forming an almost impenetrable nested acronym.
8

"Facing Government...", p. 90. "Facing Government...", p. 54. "Facing Government...", p. 67.

10

11

For example, in the case of the release of a new drug, the policy maker must balance the costs of delaying release for an industry that has invested heavily in R&D for developing the product, and the public safety protection assurances when a person first takes the drug hoping for a remedy.
12

Policymakers could also benefit by respecting the ways of scientists, who are often criticized as being too cautious, too mired in "details", and too encumbered with caveats, qualifications and exceptions, and fail to see the "big picture."
13

The Energy Department (DOE) has for decades refused requests by independent scientists who sought health data for epidemiology studies. After many exposed individuals have died, the Department agreed to begin releasing the data, but only as paper records, withholding available computer tapes. After resolving the dispute about releasing the computer tape compilations, the Department demanded that only "approved" researchers get access, setting up a lengthy approval process. See, Geiger, J. and D. Rush, Physicians for Social Responsibility, "Dead Reckoning: A Critical Review of the DOE's Epidemiological Research", 1992.
14

In addition to the computer tape example mentioned in the note above, data on "mixed" radioactive and hazardous chemical waste has been released in a format organized only by DOE-derived "treatability groups" rather than location. Therefore, determining how much waste is where requires months of painstaking manual regrouping.

15

Certainly some information should be legitimately classified and carefully restricted to protect national security. However, classification is sometimes used to deter political rather than military opponents. For example, test data from seismic monitoring stations established by NRDC were provided by the U.S. to the (then) Soviet government. However, when NRDC requested a copy we were rebuffed, resulting in a lengthy court battle, raising the obvious question: "from whom were the "secrets" being protected?" Another form of information restriction practiced exclusively by the Department of Energy is "Unclassified Controlled Nuclear Information", under section 148 of the Atomic Energy Act. This involves information that, by itself, is not classified, but as a practical matter is unavailable for public review.
16

"Facing Government...", p.50.

17

The report includes advocacy NGOs among the eight different types of S&T NGOs in its wisely titled "cautious typology". See Note 3 above.
18

"Facing Government...", p. 52.

19

In fact, because of the glare of scrutiny from the regularity of this accusation, the scientific staff of advocacy NGOs are perhaps more careful than others to avoid overstating findings. If the Task Force had included a representative of an advocacy NGO, it might have learned that this myth is largely unfounded in the actual day-today activities of advocacy NGOs facing chronic resource disadvantages. Realizing that government and industry will always be better funded, most advocacy NGOs guard their more important asset - credibility.
20

"Facing Government...", p. 26.

21

Press, Frank, NAS, Letters to Reps. Aspin, Dingell, and Miller, June 23, 1992. Similar letters were also addressed to DOE Secretary Watkins and Assistant Secretary Duffy, June 23, 1992.
22

A member of the NAS panel who helped draft the report later protested to the NAS President that it did not "need much elaboration." (Ewing, Rod, University of New Mexico, Letter to Frank Press, NAS, June 26, 1992.)
23

Other examples include: (1) The promotion of the Superconductor Supercollider is, however worthwhile, not merely an objective exercise in providing S&T information. (3) "Objective" University contractors have supported the Department of Energy's promotion the use of a computer model for setting environmental priorities that appears to many states, Indian tribes and citizens groups to be an attempt to usurp their sovereign rights. The National Rifle Association routinely floods legislatures with "objective scientific facts and statistics" to support its views.
24

Persons dealing with the government quickly learn that there is no useful difference between a real conflict of interest and the appearance of one.
25

"Facing Government...", p. 70.

26

the Task Force may wish to update it with the involvement of a number of S&T NGOs in a potentially landmark case before the Supreme Court determining the what technical evidence will be allowable in federal court. See Marsall, Elliot, "Supreme Court to Weigh Science", Science, 259: 588, Janurary 29, 1993.
27

See e.g., Fetter, S. et al., "Gamma Ray measurements of a Soviet Cruise Missile Warhead", Science, 248: 828, 18 May 1990; and Cochran, T.B. et al., "U.S. Nuclear Warhead Production", Ballinger Publishing Company, Cambridge, MA, 1988.

28

AAAS Award for Scientific Freedom and Responsibility, 1989; and American Physical Society Szilard Award, 1990 to NRDC.