Read an Excerpt
Better Environmental Policy Studies
How to Design and Conduct More Effective Analyses
By Lawrence Susskind, Ravi K. Jain, Andrew O. Martyniuk
ISLAND PRESSCopyright © 2001 Island Press
All rights reserved.
SETTING THE STAGE
The scene is the office of an environmental policy consultant who is also a professor at a top-ranking university. As we watch from the sidelines, the professor, hearing the phone ring, answers. We eavesdrop on the conversation.
Professor: Good Afternoon, Congressman Randolph.
Caller: Good Afternoon, Professor. As my assistant explained to you briefly, I have a small problem. I was hoping you could help me. Do you have a few minutes to talk?
Professor: Sure, of course.
Caller: As you probably know, there is increasing concern about the dangers to human health and the environment posed by chlorinated organics. In a speech I gave last week, I mentioned that it might be a good idea to ban or at least phase out all chlorinated organic compounds—a view that is advocated by a range of environmental groups like Greenpeace and the International Joint Commission, a watchdog group for the Great Lakes. I wasn't prepared for the flood of calls I got from industries in my district and constituents upset by my remarks.
Several chlorinated organics, including DDT, polychlorinated biphenyls (PCBs), and chlorofluorocarbons (CFCs), have been banned already or are being phased out. The use of chlorinated organic compounds is linked to increases in cancer rates and endocrine disruption, and they may pose serious risks to embryos and developing fetuses. But a host of arguments against phasing them out have appeared in scientific journals, the popular press, and political discourse. The comments Congressman Randolph heard might have included the following:
There are nearly 15,000 chlorinated organic compounds used today in various industries including plastics manufacturing, solvents, pharmaceuticals, pesticides, and paper and pulp bleaching.
A panel of Michigan scientists, convened to review the International Joint Commission's decision calling for the phasing out of chlorinated organic compounds, found that there was "insufficient scientific evidence" to support the claim that chlorinated organic compounds produce environmental and health threats.
Chlorine accounts for $91 billion of gross domestic product in the United States annually and, directly or indirectly, accounts for 1.3 million jobs.
Banning an entire class of chemical compounds does not take into account the benefits that may be provided by their continued use.
The conversation continues:
Professor: Very interesting. It sounds like you've gotten an earful! I agree; there's certainly a wide range of opinions on this subject.
Caller: Well, there's a lot more. Many of the people who called claimed that a ban on chlorinated organic compounds would be irresponsible and uninformed. My staff, though, keeps assuring me that there's documentation to prove the dangers posed by these compounds.
I know that we need to balance various interests when reaching a decision on a contentious issue. And the well-being of my constituents, indeed of human health in general, must take precedence over everything else.
So, my problem is that I need credible support for the position I have taken—a position I believe in.
Professor: I see your point. Just how can I be of help?
Caller: I would like you to put together a report indicating that there really is scientific support for my position, namely, that the continued use of chlorinated organic compounds poses a risk to human health and the environment and that a ban or a phase-out makes sense.
Professor: Hmm. I am not sure I can help. And even if I could, I'm not sure that's the right way to proceed. You have already come to a conclusion. If you simply need support for your position, then I'm not the person you need. Be aware, though, that such a report will probably not be any more credible than all the other reports, articles, and studies your staff has already looked at on the other side of the chlorinated organic compound debate.
In my view, merely commissioning one more advisory piece will just waste a lot of money. In addition, the results of such advocacy studies are often misappropriated or misunderstood, thereby making it even harder to get the real facts out.
Caller: So, Professor, what would you recommend that I do?
What should the congressman do? First, like any public official, he'll have to decide whether he is locked into a particular policy position before conducting a full-fledged environmental policy study. He needs to consider, for example, if he would support a ban or a phase-out if he learned that the risks to human health and the environment were minimal while the benefits to industry were enormous (in terms of jobs, availability of new drugs, etc.). Whether that's the case or not, only the right kind of environmental policy study would leave room for that kind of counterintuitive result.
What should the professor do? We have often told callers such as the congressman that, if your mind is already made up, we can't help you. No doubt the congressman could find some consultant happy to produce the report he's looking for. Our goal, on the other hand, is to craft studies that can be used as a basis for sound environmental policy making.
How can such studies be crafted in a charged political context like the one introduced by Congressman Randolph? That is the subject of this book. Among the principal points here is that conventional policy studies are often not up to the task. We begin by pointing out the shortcomings of the conventional approach to preparing environmental studies, particularly at the national level. We then offer a different approach, one that builds on what we think were the most useful environmental policy studies completed over the past few decades. These were studies identified by legislators like Randolph, along with government officials, industry representatives, environmental action groups, and others.
CONDUCTING POLICY STUDIES
Environmental policy studies may be undertaken in many different contexts. At the national level, a study might be requested by Congress when it is considering a new piece of legislation. Government agencies often conduct reviews of their rules and policies as a condition of their funding or in response to a request from the current administration. Regardless of whether the policy is to be carried out at the local, state, national, or international level, the issue may be highly charged, and the policy analyst may find it difficult to be heard among the competing interests at the table. The analyst must fulfill the terms of his or her agreement with the study sponsor (often one of the interests at the table) and yet remain independent and credible. The way in which the analyst factors these complications into study design often makes the difference between having an impact and accomplishing nothing at all.
Successful environmental policy studies help to shape the thinking of key policy workers and give them the confidence that they are doing the right thing, regardless of their initial thinking on the subject. Such studies share certain characteristics in that they
define the problem in a helpful fashion,
describe the full range of policy options,
help to overcome agency resistance to change,
provide opportunities for engaging interested stakeholders,
enhance the legitimacy of whatever action follows, and
facilitate setting resource allocation priorities.
In the case at hand, Congressman Randolph might think he needs only to enhance the legitimacy of particular actions. In fact, the other characteristics are equally important to him. Defining the problem correctly is critical. Is the problem of chlorinated organic compounds a problem of human health? Or is it one of jobs, economic growth, and broader societal benefit? Is it a combination of these factors? Or neither? Does it make sense to look at only one industry such as pulp and paper, or would it be best to look at all industries that use a particular chemical compound?
Second, the congressman thinks he has only two policy options: ban the use of chlorinated organic compounds, or do nothing. There may be other policy options he hasn't considered. For example, it might make sense to do a risk benefit analysis for each compound before deciding whether to ban it.
Third, some agencies may already have positions on the issue. A good environmental policy study should be structured in a way that can soften agency resistance to change.
Fourth, there are a lot of stakeholders who will be affected by a decision concerning the use of the compounds. One goal of a good environmental policy study is to involve these groups in organizing the research effort.
As we have already discussed, there are many studies that have their basis in positions for or against the continued use of chlorinated organic compounds. Yet, as the calls to the congressman's office indicate, these studies are viewed skeptically by people who don't like the recommendations they contain. It is thus crucial that a study enhance the legitimacy of whatever policy recommendations finally emerge; otherwise, the policy study probably isn't worth doing. On the other hand, if it's done right, a policy study can help set resource priorities. For example, should Congress spend money on a risk–benefit analysis for each chlorinated organic compound? Is it desirable to invest heavily in monitoring the impacts of these compounds in the decades ahead?
If the congressman is to have any hope of success, he must be extremely careful when assigning responsibility for the study or selecting experts to help. He needs to choose experts who enhance credibility and who bring a range of backgrounds to the issue. Experts also need to be familiar with parallel research efforts and be willing to seek peer review and advice.
Each consideration should play a role in deciding who should help with an environmental policy study. But there are other considerations as well. A second thing to keep in mind is how the study should be conducted. Options include:
Engaging a major research organization
Involving stakeholder groups
Appointing a commission or task force
Hiring experts directly
Doing the study in-house (with a federal agency)
Then, of course, there is the question of how to interpret the study results. In other words, what institutional auspices will be used to analyze and interpret study results to ensure that the American people will learn the most from the study.
Caller: Okay, so what you are saying is that I need to address a series of organizational concerns. I suppose you cover these in your book?
Professor: Well, actually, yes. Look at the fifth chapter when you get the book. I go into some detail about the different tools and techniques of policy analysis. I compare the value of systems analysis, decision theory, and other new approaches to internal stakeholder participation in policy analysis. There may be someone on your staff particularly interested in this methodological discussion.
Caller: Good! At least you're not expecting me to go back to school. Well, that's a lot to think about. I'm not sure I can digest everything you've said. However, I am interested in looking at the book.
Professor: I'll have one in the mail to you this afternoon. Feel free to call if you or your staff have questions.
Caller: Thanks. I'll be in touch. By the way, next time you're in DC, drop by.
If we did give a course for members of Congress, we would use Chapter 6 of this book. It reviews the three basic sets of methods and their advantages and disadvantages:
They have all been used successfully in the past. When problems lend themselves to quantification, the analytical approach may be best. However, if the study is focusing more on possible changes in policy, a rhetorical approach, which focuses on persuasion or advocacy, may be more appropriate. Finally, process methods may be most appropriate when there are fewer stakeholders or where there is a need to build consensus among affected groups.
Our hope is that legislators like Congressman Randolph avail themselves of the real and meaningful contributions that a properly conducted policy study can make. But, as we'll see, that is not the only way a policy study can be sponsored and have an impact. As you read this book, continued reference will be made to the issues raised by Congressman Randolph about a possible ban on chlorinated organic compounds. While we do not offer a complete analysis of the policy options surrounding their particular issue, we try to show how and why a "good" environmental policy study on such an issue ought to be done.CHAPTER 2
The Conventional Approach to Environmental Policy Studies
As the chlorine controversy described in Chapter 1 suggests, numerous policy studies have been undertaken on this and many other controversial environmental issues. Studies on the same subject often come to widely varying conclusions on the human health and environmental risks posed by chlorine and other substances. Lobbyists, activists, and others have tried to use such studies to sway public opinion and to convince policy makers to take action or not to move forward. Finally, studies of this kind have, in fact, resulted in the adoption of certain national policies which, in turn, have become the subject of additional policy studies. Why so many studies? Why so many different results?
In part, the large number of studies conducted on the risks posed by chlorine and other substances owe their existence to the way in which environmental policy studies are traditionally organized and carried out. The conventional approach to such studies virtually guarantees controversy. In the following sections, the conventional approach to conducting environmental policy studies is reviewed with a special emphasis on the various procedural choices that must be made at each step along the way. Next, the connection between policy studies and policy making is discussed. The chapter concludes with a critical review of the conventional approach to conducting environmental policy studies.
HOW ENVIRONMENTAL POLICY STUDIES ARE USUALLY DONE
The conventional approach appears quite logical and mirrors related activities in other sectors of the business or consulting world. However, as mentioned earlier, the results of this process leave decision makers with the unenviable task of sorting through conflicting findings from which they must derive policy advice. What we call the conventional approach can be viewed as a six stage process:
1. Define the issue and suggest a method of analysis.
2. Choose a consultant.
3. Write a contract.
4. Perform the analysis.
5. Submit a draft report for comment.
6. Produce and disseminate a final report.
As with any other business contract, the individual or group seeking an environmental policy study (to answer a question or support a decision they have already made) first narrows the issue or issues they want to address and then selects a method of analysis. The ultimate conclusion may also be stipulated. This information is then embodied in an offer, or, if the government is involved, a request for proposals (RFP). Results, findings, and recommendations are reviewed before a final report is submitted. Ultimately, a final report meeting the contract requirements is submitted and forms the basis for a decision or a policy choice.
Define the Issue
The issue may be construed as broadly or as narrowly as the sponsor wishes. For example, Congressman Randolph (whom we met in Chapter 1) could define the issue as the desirability of banning the use of chlorine. Alternatively, the congressman might define the issue in terms of limiting or reducing the use of chlorine in a particular industry (e.g., paper and pulp). Or, the congressman might define the policy question in terms of the feasibility of a shift to safer chlorine substitutes to replace the compounds used by industry at present. Methods for defining a problem include back-of-the-envelope calculations, quick decision analysis, political analysis, issue papers, and creation of valid operational definitions (Patton & Sawicki, 1993). These can all be applied by the sponsor to refine an issue (or a conclusion), or by a consultant hired to assist in undertaking the policy analysis that has been requested. The findings may be preordained, particularly if the sponsoring organization has its mind made up in advance, and hires a consultant who shares its bias. It is therefore no surprise that many environmental policy studies, all claiming legitimacy, advocate different, or even opposite, positions by the time they are completed. The outcome is a function of the way the issue is framed at the outset.
Choose a Consultant
An individual or group can be selected to perform the analysis. There are several factors that come into play in making this decision. First, one often seeks the best consultant possible for the money, the tacit trade-off being that an individual who is "better" (e.g., more widely known, more highly respected) will add credibility to the findings and, ultimately, to what is recommended. Often, however, study sponsors tend to favor a consultant whose point of view or leanings are already known. Although such an individual or group may not have as stellar a reputation, they are more likely to produce a report that will be received favorably by the sponsor. Finally, consultants may be selected based on their experience in dealing with controversial policy issues. The sponsor may put a premium on having a consultant who knows how to interact with the press and how to function effectively in the midst of controversy. In many ways, this sort of selection process mirrors the use of scientific experts in civil litigation. The expert would not be hired unless he or she could be counted on to produce findings that supported the position of the hiring party. Thus, as with the framing of the issues, choice of consultant plays a role in generating the disparate results produced by environmental policy studies covering similar issues and using equivalent data.
Excerpted from Better Environmental Policy Studies by Lawrence Susskind, Ravi K. Jain, Andrew O. Martyniuk. Copyright © 2001 Island Press. Excerpted by permission of ISLAND PRESS.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.