Five-Year Program Review Regional Monitoring Program for Trace Substances in the San Francisco Estuary |
| Panel: |
|
Contents 3.0 Data Analysis and Study Design 4.0 Quality Assurance and Data/Information Management and Transfer 5.0 Organizational Relationships and Capacities 8.0 Conclusions and Recommendations for Implementation |
Executive Summary The Regional Monitoring Program for Trace Substances in the San Francisco Estuary (RMP) has successfully produced high-quality data on chemical contaminants and their toxicity throughout San Francisco Bay. Since its inception in 1993, it has combined shared support, direction, and participation by regulatory agencies and regulated organizations/industries in a model of collective responsibility. As a result, it is developing an expanding database of information that has helped to address important decision-making needs of regulatory agencies and other program Participants. This report presents the findings and recommendations of an in-depth review of the RMP carried out during its fifth year of operation. This review was an integral part of the program's initial five-year plan and was carried out by a panel of nationally recognized experts in a range of fields. Its objectives were to:
The RMP has faithfully addressed its guiding objectives and has achieved notable successes during its first five years of operation. These include:
As a result of these successes, the Review Panel found widespread support for the RMP, many instances of its usefulness, and a firm commitment that it should be continued for at least another five years. The Review Panel also found, however, that these very successes, along with five years' experience and the benefit of hindsight, have raised serious issues that must be addressed if the RMP is to fulfill its potential. The Review Panel believes that complex programs such as the RMP must continue to evolve in response to their users' needs if they are to avoid the "monitoring trap" of simply collecting data for its own sake. In the RMP's case, two core themes consistently arose in the evaluations the Panel carried out from a variety of perspectives (basic objectives, study design, data analysis, information management, organizational dynamics, and management). The first theme is the need for more detailed definitions of all aspects of the RMP, in particular of
The RMP's original objectives provided effective guidance during the program's early years. However, at present they are not sufficiently detailed or specific enough to effectively focus the program's efforts on management's key information needs. As a result, much of the current data analysis, interpretation, and reporting is diffuse and not particularly relevant. Similarly, the program's commitment to consensus-based management has helped build an important degree of involvement and commitment on the part of all parties to the RMP. On the other hand, it has also resulted in an inability to directly address important issues, such as developing more detailed objectives, where there is disagreement among some of the parties. The Panel recommended that the RMP make it a high priority to address the issues listed above as part of developing a new five-year plan. The second theme is the need for the RMP to broaden its scientific horizons in order to increase the usefulness of its results in decision making. The Panel strongly recommended that the RMP undertake modeling and analysis to place the RMP data in the context of other data from San Francisco Bay. In particular, historical data can provide a larger perspective within which to interpret the relatively short time series of data developed to date by the RMP. These other datasets represent a valuable resource that is currently being under-utilized. In addition, the Panel recommended that the RMP address a wider range of fundamental scientific issues that are key to any attempts to interpret the implications of the RMP's monitoring data. These issues include such questions as the annual input of key pollutants to the Bay, the response of the Bay system to past reductions in pollutant input, and the relationship between observed patterns and trends of key pollutants and various kinds of sources, both human and natural. The Review Panel believes that such issues are not unique to the RMP but are challenges that typically face complex environmental monitoring and management programs. The Panel further believes that the parties to the RMP have the commitment, understanding, and ability to successfully meet these challenges and to continue to make the RMP a model of cooperative environmental problem solving.
1.0 Introduction The Regional Monitoring Program for Trace Substances in the San Francisco Estuary (RMP) is an innovative and successful program aimed at collecting data on chemical contaminants in the water and sediments of San Francisco Bay, evaluating the in situ bioaccumulation potential of trace contaminants by biota in the Bay, and determining the potential for water and sediments of San Francisco Bay to cause toxicity in bioassay organisms. The RMP has combined shared support, direction, and participation by regulatory agencies and regulated organizations and industries in a trend setting model for collective responsibility in assessing the overall condition of San Francisco Bay. Such a model for collaboration and cooperation deserves national and international recognition. Such a partnership does not, to our knowledge, exist anywhere else in the nation or the world.
1.1 A Planned Comprehensive Review Preliminary Bay-wide studies of contaminant concentrations began in 1989 with seed money from the USEPA. Additional Bay-wide sampling was undertaken beginning in 1991 within the Bay Protection and Toxic Clean-up Program. The RMP was formally implemented in 1993. At the time of this writing, the RMP is in the fifth year of its routine Bay-wide core sampling program. A variety of special and pilot studies sponsored by the RMP have been carried out for shorter periods. The RMP has evolved dramatically through these five years. It has developed an expanding base of high-quality data on samples from stations located in mid-bay (the spine) and generally away from the confounding influences of point-sources of environmental pollutants. In addition, valuable information from special and pilot studies has assisted in refining understanding of the data from the core program and suggest potential new areas for eventual inclusion in the RMP. A comprehensive program review was anticipated as part of the initial RMP design, and was set to occur in the fifth year of the study. Notwithstanding this planned review, the RMP and the study plan have been subject to constant review and evaluation between 1993 and 1997. Regular meetings of the RMP Steering Committee (SC) and occasional meetings of the SFEI Committee of Scientific Advisors (CSA) have watched over the program and have recommended changes and improvements. Program development and evolution have also been influenced by the results of pilot and special studies. The RMP Five-Year Review reported herein, however, is the first time the program has been comprehensively and formally evaluated by independent scientists and specialists from outside the San Francisco Bay area. 1.2 Review Objectives The objectives of the Five-Year Review were to determine the successes and shortcomings of the overall program during the first five years of its existence by examining the structure and performance of the RMP and its staff, contractors, and administrative infrastructure. The review was intended to recommend which parts of the existing program should be retained or amplified to maintain performance at a high level, and to suggest changes, additions, or deletions that might improve the program. To that end, a panel of seven experienced individuals was contracted and assembled to examine all aspects of the RMP and to interview RMP participants familiar with its scientific, logistic, administrative, management, planning, and quality assurance aspects. The review process was focused on providing answers to general questions, such as:
This planned report, answering these and more specific questions and containing specific recommendations, is hereby submitted to the RMP Steering Committee for evaluation and, where possible, for implementation. 1.3 The Review Process The review process included:
Members of the RMP Five-Year Review Panel were:
The review was facilitated by two coordinators, Drs. Brock Bernstein and Joseph O'Connor. 1.4 Overview of Report Scope and Structure The final report for the RMP Five-Year Review addresses all aspects of the project. It consists of seven chapters in addition to the Introduction and Executive Summary. These chapters address objectives, data analysis and study design, quality assurance and data/information management and transfer, organizational dynamics, management, resources, and implementation. Each chapter evaluates the RMP from a different perspective, draws conclusions about the successes and/or shortcomings within each area of interest, and makes appropriate recommendations. As described more fully in the Executive Summary, it is significant that the same core themes consistently arose from the separate analyses from different perspectives. It is the overall conclusion of the Review Panel that the RMP, after five years of operation, is a highly successful program that is meeting its original objectives and is of great value to its participants. The Review Panel notes that the program has not remained static over the first five years of operation. Nor should it! Approaches, methods, analyses, and questions of concern have all evolved with the gradual accumulation of data and experience. In all technical areas, the direction of the RMP's evolution has been positive. The RMP of 1997 has matured to the point where it would benefit from a collaborative effort among all participants to provide more detailed statements of the project's overall goals; an expanded, clearer set of program objectives; a more precise set of scientific questions to be answered; and a more distinct definition of roles, responsibilities, and decision-making processes.
2.0 Program Objectives Summary: The RMP's original objectives have provided effective guidance during the program's first five years and have been largely met by the existing sampling and data analysis designs. Recently, some parties to the RMP have suggested that the program's objectives should be reconsidered and modified in order to address a broader set of issues. This has created a healthy tension within the RMP. The Review Panel believes that the RMP's objectives should evolve over time and that an expanded set of objectives is needed to better integrate the RMP into the environmental management system for the Bay. However, the Review Panel also believes that a successful reevaluation of the RMP's objectives depends on a better and clearer definition of the roles and responsibilities of the various parties to the RMP. The impetus for the Regional Monitoring Program for Trace Substances (RMP) was the Comprehensive Conservation and Management Plan (CCMP) produced in 1993 by the San Francisco Bay Estuary Project. Specifically, the CCMP identified as one of its objectives:
The CCMP identifies as an action item the development and implementation of a Regional Monitoring Strategy (RMS) to integrate and expand on existing efforts and eventually be part of a comprehensive Regional Monitoring Program. The RMP for Trace Substances was created as one component of the anticipated Regional Monitoring Program through the leadership of the San Francisco Bay Regional Water Quality Control Board (Regional Board), which encouraged the participation and financial support of sponsors (regulated dischargers, public facilities, and industries). These sponsors are referred to as RMP Participants. The RMP has operated under formal objectives (Box 1) since its beginning in 1993. These objectives were developed by the staff of the Regional Board, RMP Participants, and San Francisco Bay Estuary Institute (SFEI) staff. Because of the regulatory issues surrounding discharge permits, the RMP objectives have focused principally on contaminant concentrations in Bay water, sediments, and bivalve tissues and on biological responses relevant to determining water and sediment quality as revealed through bioassays. 2.1 Meeting the Objectives A review of the activities and products of the RMP shows that, to a very large degree, it has been faithful in addressing its original objectives, both implicitly and explicitly. RMP Participants expressed satisfaction with the way SFEI and the prime contractor (Applied Marine Sciences of Livermore CA) have kept the RMP focused on meeting these objectives. The high quality measurements that have been made are sufficient to quantify existing conditions in the center of the Bay. Analyses of data completed through 1995 (SFEI 1997b) attempt to describe spatial and seasonal patterns that might be discernible from the data (see Chapter 3 for discussion on data analysis and design). Comparisons have been made among years, although too few years of data are available to assess interannual variability or long-term trends. The data on contaminant concentrations allow determination of the extent to which open Bay waters are in compliance with the numerical criteria for water quality established in the Basin Plan. A database on water and sediment quality has been developed and maintained. RMP data are available to the environmental community directly from SFEI or through the RMP/SFEI Website. Although the database is generally compatible with data from other studies in other parts of the Bay, some comparability problems inevitably occur as a result of different sampling and analytical protocols. Interpretative reports by the RMP have attempted to reconcile these differences in comparisons. The RMP has clearly addressed Objective 5 by providing a database compatible with these other studies. However, it has been less successful in fostering integrative analyses involving, for example, the Interagency Ecological Program, Mussel Watch, or dredged material disposal monitoring data. 2.2 Pilot and Special Studies In addition to the routine monitoring measurements, RMP support funds pilot studies and special studies. Pilot studies are intended to assess methods under consideration for incorporation into the RMP program. Special studies are aimed at improving interpretation of RMP data (Box 2). Pilot studies and special studies have helped to improve analysis, enhance the comparability of results, enhance the relevance and sensitivity of bioassay tests, and examine the distribution of contaminants in some areas (the South Bay and wetlands) in greater detail. RMP Participants do not all agree on the value or the relevance of the pilot and special studies. On the one hand, many Participants believe that pilot and special studies have been effective in exploring new methods or approaches and assisting interpretation of monitoring data. On the other hand, some Participants are concerned as to whether some pilot and special studies depart from the original RMP objectives. Some special studies deal with issues of interest to some Participants and not others. And, the upshot of such conflict is that opinions on pilot and special studies differ depending on the perspective of the Participant. For example, the Regional Board's interest in surveys of toxicants in fish was not shared by many discharger Participants. However, agreement was eventually reached that a special study on toxicants in fishes would be carried out, although at a reduced frequency. The RMP Technical Review and Steering Committees (see Chapter 5) have mechanisms in place for resolving differences regarding the scope and relevance of proposed pilot and special studies. The Review panel believes (as noted in Chapter 5) that the current consensus-based mechanism for resolving such disagreements would profit immensely from clarifying both the roles and responsibilities of RMP Participants as well as decision-making processes. The planning, evaluation, and funding for pilot and special studies would also profit from attention to the recommendations in Chapters 3 and 5 regarding improving the overall focus of the study design. 2.3 Meeting User Needs The complex nature of the RMP, with different entities responsible for design development, contracting, logistics, sampling, analysis, testing, and reporting made it impossible to perform a unit-cost evaluation. The RMP Participants and representatives of federal and state regulatory agencies (USEPA, Corps of Engineers, and the Regional Board) expressed satisfaction with the quality and usefulness of the monitoring program results and offered numerous examples of how RMP data have been used. Participants agreed that they are receiving a quality product for a reasonable cost. The Review Panel concurs in this conclusion. However, some representatives of the dredging community questioned the utility of RMP results in addressing problems associated with regulatory testing requirements for dredged material disposal. Regional Board staff use RMP results in a variety of ways. They help to determine discharge permit requirements and are being used to establish background sediment contaminant concentrations and in the development of sediment quality criteria. However, the full extent to which RMP results are used by the Regional Board was not apparent to the Review Panel, and is not always clear to the regulated dischargers who support the RMP. The differences in opinion and perception on this issue reflect, to a large extent, the absence of clear detail about how RMP data should be analyzed and on what questions/problems these analyses should focus (see Chapter 3 for more discussion). An exception is determining compliance with numeric Basin Plan objectives. One incentive for the Participants to continue supporting the program is the expectation that their continued participation will be rewarded with reduced self-monitoring requirements specified in discharge permits issued by the Regional Board. Another incentive is that continued participation will be rewarded with increased regulatory flexibility. Some Participants believe these benefits are being realized, while others believe that they have not yet been fully achieved. It is significant that the Participants who have reaped such benefits (e.g., POTWs, the larger dredgers) attributed this to their own in-house ability to use RMP data to address and argue questions of specific interest. As discussed further below, this reflects an implicit assumption in the RMP's objectives that the RMP's function does not extend to application of RMP data to problems of specific regulatory interest to the Participants. 2.4 Evolving Objectives The RMP for Trace Substances addresses only a fraction of the activities specified for the Regional Monitoring Program that the CCMP called for (San Francisco Estuary Project, 1993: p. 184). The RMP does not address the broader issues in the CCMP mission statements that include restorating and protecting diverse, balanced, and healthy populations of Bay organisms; protecting beneficial uses; and eliminating and preventing pollution at its source (San Francisco Estuary Project, 1993: p. 28). For example, the original RMP objectives (see Box 1) do not include the development of inventories of contaminants or any activities related to identifying sources of contaminants. Both might be helpful in linking RMP monitoring results to pollution elimination and prevention, determining effects of contaminants on populations of Bay organisms (other than as reflected in bioassays), and assessing effects on beneficial uses of the Bay's resources. As a result, several parties to the RMP, including SFEI and some Participants, believe that the RMP's objectives (Box 1) should be modified. In order to structure this discussion, Dr. Bruce Thompson of SFEI (1997a) proposed a new Objectives Framework for the RMP that reflects a series of internal conversations among RMP Participants. The proposed Framework, currently being considered by the Steering Committee, would modify the original objectives to include the statements shown in Box 3. The proposed Framework is based on four tiered steps:
The Steering Committee has provided little comment on the proposed Framework. However, several concerns about this expansion in scope were voiced to the Review Panel. First, the RMP Participants are very concerned about the growing costs of the existing program and the greater potential costs of a program with broader objectives. Second, there is concern that the RMP might lose its focus on contaminants and, instead, emphasize broader concerns about ecosystem and human health that may have little relevance to regulated discharges. However, Participants are receptive to efforts that might determine sources of contamination, particularly insofar as they might demonstrate that riverine, atmospheric, and nonpoint sources contribute more contaminants to the Bay than point-source waste discharges. Third, RMP Participants are, to varying degrees, concerned about broadening the interpretation and dissemination of results, fearing that additional interpretation has the potential for going past scientific results into the realm of policy and advocacy. Regional Board staff have also expressed concern about expanded RMP objectives. They believe that it is the role and responsibility of the Regional Board to interpret monitoring results with regard to regulatory actions. They are concerned that expanding biological measurements to assess ecosystem health may produce data that are difficult to interpret, and for which there are no existing criteria. It is difficult to judge the degree of the Regional Board staff's concern because of transition of staff responsible for interfacing with the RMP and the fact that Regional Board staff members have somewhat different views on this subject. Overall, however, Regional Board Staff are sensitive to any changes in the RMP's objectives that blur the boundary between technical interpretation of the data and the regulatory and policy responsibilities reserved to the Board. In addition to these concerns expressed by those directly involved in the RMP, SFEI's Committee of Scientific Advisors (CSA) (Swartz et al., 1995) reviewed the RMP and stressed the need for a strategy to guide the RMP into the future. This would help the RMP avoid the fate of many monitoring programs, i.e., collecting data that are never properly analyzed or used effectively in management decisions. The CSA stressed the importance of synthesis and interpretation, the need for direct biological measurements of effects to complement chemistry and toxicity tests, and summaries of contaminant inputs. The CSA endorsed the continued focus of the RMP on contaminants, but recommended better integration of the RMP with critical parts of other monitoring programs. These recommendations correspond with the Review Panel's recommendations in Chapters 3 and 5 on the need for more explicit links between program objectives, questions of specific concern to decision makers, and the sampling and analysis designs. The mismatch between the limited objectives for the RMP (Box 1) and the desire in some quarters for broader monitoring and assessment objectives creates tension among the RMP Participants, the Regional Board, and SFEI. The Review Panel stresses that such tension can be healthy (Chapter 5) if it leads all parties involved to reevaluate their expectations and roles. In fact, such tension is inevitable, because the nature of environmental problem solving precludes drawing perfectly clear boundaries between monitoring, analysis, interpretation, and objective setting. However, as discussed in more detail in Chapter 5, such tension can be counterproductive when there is no well-defined process for addressing and resolving conflicts. The Review Panel identified three areas of tension in particular:
2.5 Role of Monitoring in Environmental Management As pointed out by the National Research Council (1990), monitoring should be an integral component of an environmental management system. Monitoring should be effectively linked with research and modeling to inform managers and to help them answer management questions. The NRC report outlined a conceptual model relating the technical design, implementation, and interpretation of monitoring programs to environmental quality objectives and decision making. Similarly, the subsequent NRC (1993) review of wastewater management in urban areas described the role of monitoring as an important element in the integrated management of coastal issues. The RMP set out to provide data for two general purposes: 1) to determine whether water quality in the Bay is in compliance with the Basin Plan, and 2) for broader environmental management. However, neither the original objectives (Box 1) nor the study design were developed to ensure that regional monitoring serves as an integral element of environmental management (see Chapter 3 for specific examples related to data analysis and study design). The proposed program objectives (Box 2) place greater emphasis on impairment of beneficial uses; sources of contamination; and synthesis, interpretation, and reporting. By doing so, they seek to integrate the RMP in the environmental management systems for San Francisco Bay. The Review Panel supports such efforts. Modified objectives would provide a basis for formulating more appropriate questions to guide study design, data analysis, and interpretation (see Chapter 3). The Objectives Frameworks discussion paper (SFEI, 1997) can serve as a point of departure for this process. However, The Panel believes that the success of such discussions will depend largely upon implementing the Panel's recommendations (Chapters 3, 5, and 6) regarding better defining the roles and responsibilities of all parties to the RMP. 2.6 Recommendations Based on the foregoing considerations, the Review Panel makes the following recommendations regarding the objectives of the RMP:
3.0 Data Analysis and Study Design Summary: To date, the RMP has produced high quality data describing the present state of the Bay. Because of their quality, these data will remain useful into the indefinite future as a description of conditions in certain portions of the Bay. However, there is a disconnect between the questions of most interest to sponsors and the descriptions of conditions produced so far by the RMP. This is reflected in the character of the data analysis approaches used by the RMP, which are mostly unfocused descriptive data presentation, relatively little focused summarization, and hypothesis testing that is oversimplified or of little relevance to decision making. The Review Panel found that this stems largely from the absence of clearly defined, specific questions that can link monitoring to decision making and that can be addressed with the appropriate range of analysis and modeling approaches. The Review Panel believes, quite strongly, that developing the ability to frame clearer questions and to better articulate management's needs for scientific information is an organizational issue involving the entire RMP. Once such questions have been developed, the overall study design can then be reevaluated in terms of its ability to help answer them, and the Panel did identify aspects of the study design that might be streamlined. At present, however, the absence of detailed guiding questions and the lack of any formal justification for the existing design restricted the Panel's ability to complete such an evaluation. 3.1 Introduction The fundamental justification for the RMP is to provide information to better manage the San Francisco Bay ecosystem and protect beneficial uses. The National Research Council (NRC, 1990), in its nationwide study of marine environmental monitoring, noted that monitoring programs must recognize the difference between data and information in order to provide useful input to such management goals. They emphasized that "... raw data collected in a monitoring program frequently do not [alone] directly address public concerns or the information needs of decision makers." Rather, monitoring data must be gathered, analyzed, and interpreted with the intention of addressing specific management objectives and related scientific questions or problems. While there is a wide range of approaches to crafting policy and making decisions about environmental issues, the successful ones all share this emphasis on the development of appropriate monitoring and other scientific data that can direct, set limits on, or otherwise inform management decision making. Monitoring data come from well-managed sampling and analysis programs, but useful information can only result from sampling/analysis programs whose design reflects a clear perception of what questions are important to decision making. In the words of Drucker (in NRC, 1990), "Information is data endowed with relevance and purpose. Converting data into information thus requires knowledge." The following sections review the RMP's data analysis methods and study design in terms of their ability to create useful information. They also identify specific areas for improvement, particularly in better defining detailed questions to guide data analysis, synthesis, and interpretation. 3.2 Data Analysis Analyses in RMP reports are of two kinds, descriptive and inferential. Understanding the differences between these is important for assessing the relevance and usefulness of RMP analyses and for identifying opportunities for improving these. Descriptive analyses display or summarize the data without assessing their reliability or attempting to target the description at a specific question in the real world. Inferential analyses usually involve formal hypothesis testing and are appropriate only for addressing specific, well-formed questions. Inferential analyses also necessarily involve assumptions about sources of variability and uncertainty in the data, issues intimately connected to the monitoring design itself.
3.2.1 Description - Unfocused and Focused 3.2.1.1 Unfocused Description - Well-presented unfocused descriptions are standard and reasonable. They are an archive of the year's results, and allow any reader to make or check theories or suspected patterns for any variable of interest. They make it possible, in principle, for any reader to detect anomalous results, whether due to a data reporting error or to a change in pattern that requires attention from regulators or researchers. The majority of RMP analyses to date consist of unsummarized, unfocused description of raw data. By unsummarized we mean that each data point appears explicitly, and there is no summarization except for the processing and averaging used in chemical analysis. By unfocused we mean that the descriptions provided are not aimed at answering any specific question or elucidating any particular pattern. Bar charts give values for stations at a particular time, and time series plots give values for times at particular stations. There are a few tables such as those giving stations and times where sample measurements exceeded Basin Plan guidelines. Other bar charts, plots, and tables use a small amount of summarization to combine stations at a given time, usually by giving the range of raw values. Some of the plots describing toxicity testing results use standard errors, but these measure only sampling error, due to variation between animals, and not between places or times. Some of these plots and tables need more careful presentation. It is confusing to present data by "Cruise number" instead of by the more informative identifier "date." In some articles on toxicity testing, error bars in plots are not described, and calculation methods are not given. It should be stated whether error bars are standard deviations, standard errors, or ranges. In some tables, sample sizes were not given. Presentation of time plots is partly a matter of taste, but spikes as in Figs. 49-54, pp. 55-65 of the 1995 Report are distracting and decrease the data-to-ink ratio. More careful proofreading of tables would be helpful to ensure consistency of format and readability. Such raw descriptions, no matter how well presented, are, however, mainly useful as an archive since they do not give an overall picture of the Bay with respect to any variable. There are too few data for a synoptic picture. But, paradoxically, there are also too many data for the human mind, unaided, to construct an informative global picture. It is similar to trying to imagine a portrait based on a list giving the coloring of each square millimeter. The problems inherent in unfocused description, and those pertaining to the style of presentation, make such analyses of only marginal use for decision-making. They are even less useful for planning or designing future sampling. These activities require more focused summarization, inference, and modeling. The Review Panel believes that the problems associated with unfocused descriptive analysis should decline as the RMP database expands to support more sophisticated analysis. 3.2.1.2 Focused Description - The goal of focused descriptive analyses should be to find ways to display the data so that they reveal or suggest patterns obscured by more simple unfocused descriptive analyses. Thus, the RMP should, at a minimum, concentrate on filling in the empty spaces in maps of the Bay with interpolation and other methods, combining values into comprehensible patterns, and smoothing the less reliable values. Displays of such regional patterns are, however, rare in the RMP reports; they usually appear not in regular reports by SFEI personnel but in special studies or reports prepared by consultants or by USGS. This may be due partly to the amount of spatial filling in required, since the USGS sampling is much more intensive both spatially and temporally. But it might also reflect SFEI's sensitivity to concerns about the degree to which it interprets the monitoring results (see also Chapters 2 and 5). The Review Panel believes it would be useful to experiment with choices of display methods (symbols, colors, shading, connecting lines, etc.) and with finding ways to include more information without loss of clarity. Contouring, shading, or using estimates or interpolated values at unsampled points could be effective. A combination of such methods could display spatial variation in two variables at once. There are several helpful books and papers available, since this is an active area of statistical research. A report could well contain many such displays, given the large numbers of variables (and variable combinations) of concern, and this could markedly improve the usefulness of the Annual Reports. It may be that the most read parts of the Annual Reports are the Executive Summary and parts of the Conclusions (e.g., the Summary of Estuary Condition). These are focused summaries that usually avoid judgments by simply adding the numbers of exceedances of various official standards and guidelines at each station or group of stations. They are useful in two ways. First, they suggest the contaminants of most concern, and second, they focus attention on the appropriate data sets and reports. However, they leave out information, as any summary must. And these summaries may leave out important items such as whether an exceedance is great or marginal, the reliability of the determination, or how well-based or appropriate to the Bay the standard is. Such summaries are therefore most useful when placed in the context of attempts to answer the kinds of more concrete questions important to decision making (see below, this Chapter; also Chapter 2 relating to the development of a new objectives framework). The Review Panel feels that the lack of synthesis and focused description hinders users' ability to comprehend the RMP's results. As suggested by interviews with Participants, Regional Board Staff, and scientists making use of the RMP reports, a greater degree of data synthesis is recommended, accompanied by greater attention to focused, descriptive analysis. 3.2.2 Inference Inferential analyses attempt to draw clear conclusions about specific suppositions or hypotheses, accompanied by a measure of their reliability. These analyses are framed as a related set of conceptual, mathematical, and statistical models. They thus require that the question being asked be formulated very specifically so that the relevance and appropriateness of these models can be evaluated. This can be a difficult task, however, and there is often a strong temptation to convert difficult inference problems into standard ones, with the result (often unintended and/or unnoticed) of simplifying or changing the question. While succumbing to this temptation avoids approximate, uncertain answers, it also leads to asking and answering questions that may be of little interest or relevance. The likelihood that inference will address the wrong question is even greater if detailed questions have not been clearly articulated ahead of time, in this case by the Regional Board and the RMP Participants. Hypothesis testing is used frequently in the RMP reports. However, with the benefit of 20/20 hindsight, the Review Panel believes that, in most cases, the hypotheses being tested are overly simplified or of little relevance to actual decision making. An example follows. Pages 44-52 in the 1995 Annual Report present results of ANOVA and pairwise t-tests to compare South, Central, and North Bay means for the total and dissolved fractions of ten elements on each of eight cruises. The form of these tests permits asking only whether these portions of the Bay were different. In fact, the question as to whether these parts of the Bay are different is rather uninteresting. Given the known contrasts in depth, kinds and amounts of riverine and urban inputs, degree of mixing, and amount of ocean exchange, it seems a foregone conclusion that these tests would find differences among the three portions of the Bay. In fact, it would have been surprising had they not. Furthermore, the form of the test only permitted asking whether these parts of the Bay were different on such and such a cruise, which is an even more limited, and less-interesting, question. Documenting such differences does accord with the RMP's objective of developing high-quality baseline data and the knowledge that such differences exist might be of interest to the Regional Board and the Participants. However, the lack of any larger scientific and management context for the analysis seriously reduces the usefulness of these results. Thus, it is not clear, for example, whether a difference of any particular size is of more interest or import, whether these differences can or should be related to sources, whether the differences are related to biological impacts of concern, or the relative importance of anthropogenic and natural processes in generating and maintaining such differences. Such weaknesses in the use of inferential analyses are not the fault of any of the organizations involved in the RMP, nor do they invalidate any of the excellent work done by the RMP. Rather, they highlight the need for more explicit questions to guide study design and data analysis as the RMP evolves beyond its early years. The Review Panel suggests that these questions might include topics such as:
The Review Panel believes that attempting to answer clearly defined, specific questions such as these is the key to creating useful information from raw monitoring data. Addressing such questions requires 1) that they be articulated in the first place, 2) that they be evaluated with a wider and more appropriate range of analysis and modeling approaches, and 3) that they be assessed within a larger context of knowledge about the processes and mechanisms in the Bay that control the input, distribution, and fate of trace substances. The Review Panel also believes, quite strongly, that developing the ability to frame clearer questions and to better articulate management's needs for scientific information is an organizational issue involving the entire RMP (see Chapter 5 for further discussion). At present, the RMP data may be too sparse for the kind of in-depth inference the Panel advocates. There are completed reports on only nine cruises and data for only twelve. This is rarely enough for inference tests that deserve to be taken seriously. This does not mean that no conclusions can be drawn or that data analysis is pointless. It does, however, emphasize the importance of integrating the RMP data with other data on historical trends, sources of inputs, and contaminant impacts. It also underscores the necessity of creating a larger conceptual framework, such as (even crude) mass balance models, that provide a means of articulating, checking, and improving assumptions and predictions about the behavior of trace substances in the Bay. At present, there is little such synthesis and integration of RMP and other data in the Annual Reports. In summary, archival analysis (unfocused description) is done well. The several kinds of focused analyses, including inference, are rarely done and are not done well. Where they are performed, investigators make up their own (often implicit) questions, especially for formal inference. The main reason for this is that there are few or no focused questions from the RMP's clients to provide the guidance needed for such analyses. 3.3 Study Design 3.3.1 Design Rationale The Review Panel could not effectively evaluate the study design because little formal rationale has been given for the RMP's design. It was particularly difficult to understand the number and placement of stations and the choices of sampling times. What rationale was available either lacked specificity, took key issues for granted, was anecdotal in nature, or provided extensive technical detail without relating this detail to explicit information needs for management. This section reviews an example of such a weakness and then poses some questions the RMP managers might address to better justify, focus, and streamline the design. An illustrative example is the choice of sampling stations along the spine of San Francisco Bay. These stations were chosen so the data could be used to interpret "temporal and spatial variability in the data without the confounding variable of contaminant input from nearby sources." This justification raises more questions than it answers. This design is useful for estimating longitudinal variability along the length of the Bay, but not spatial variability between the shore and the middle of the Bay. While such longitudinal designs are traditional in estuarine sampling programs because of the influence of salinity and other gradients, there is no explanation of how and why such gradients might be important in this case. That is to say, there has been no explicit question articulated that could guide the process of making decisions about where RMP stations might be sited. Confounding the issue, some stations that have been added to the design since 1993 are not on the spine. The rationale for including these stations was "to better monitor areas around major tributaries." This contradicts the first rationale, since tributaries are precisely those "confounding" sources of contamination avoided in the initial design that sited stations on the spine of the Bay. While both sets of stations may be perfectly reasonable, there is no clear statement as to how these two objectives relate to each other, whether the newer stations were chosen to improve the linkage between source monitoring and Baywide averages, or what kinds of analyses to establish such a linkage would be compatible with the sampling design. Nor is there documentation that explains why the spine stations are the best choice for estimating Baywide averages and trends in averages. Such averages depend on the relation between the spine and the Bay as a whole and on how the average itself is changing over time. There is a range of plausible scenarios (e.g., nonlinear relationships, changing patterns of inputs, time delays in system response to changes in inputs) that could undermine the validity of this sampling design. There seems to be an implicit assumption that data taken from the spine will have a higher signal-to-noise ratio than data taken from a more broadly distributed array of stations. However, this assumption is never evaluated, either conceptually or with data. The point of these comments is not that the spine stations are wrong, but that little justification has been given for them, either in strictly scientific terms or in terms of the kinds of more detailed questions important to the Regional Board and the Participants. Decisions about these and other key design questions require clear statements about study goals, models that organize assumptions about system behavior, estimates of variation and correlation, and more comprehensive synthesis and use of other (particularly historical) data. 3.3.2 Other Issues Tidal Cycles: Some interviewees suggested that the timing of samples relative to the tidal cycle could lead to misinterpretation of results. If a particular cruise track circles through the Bay following a rising tide, then subsequent samples could be roughly following the same packet of water as it moves in and around the Bay. It is not clear how often this might occur or what its effect might be. For example, ocean water may intrude below Bay water, and not much affect either the top one meter where water samples are taken or the sediments. Even if there is an effect, it might not corrupt trend estimation provided there is enough mixing of Bay and ocean water and sample timing is similar every year. However, if this is a problem, estimates of Bay contamination could in fact be estimating a varying weighted average of the ocean and the Bay, making trends more difficult to estimate. Power Analysis: It is customary and useful to estimate the statistical power of monitoring designs. This may be difficult for the RMP, given the many sources of variability and how little is known about them. However, the discipline of defining statistical models, identifying and estimating sources of variability, and calculating statistical power provides two important benefits. First, it forces program managers to make their assumptions explicit, an important first step in validating the soundness of the monitoring design. Second, it can provide insight into whether the design will be able to find the kinds and amounts of change that have either been predicted or are important to document. The absence of any formal power analysis of the overall RMP perhaps reflects the fact that there is no statement about the kind and amount of change the program should detect. For example, it is not clear how much of a change from year to year, nor what sort of a long-term trend the design should detect, or is hoping to detect. Without such guidance from managers, it is impossible to determine whether the trend monitoring program is technically adequate and cost-effective (see Section 3.4 on Setting Study Goals). 3.3.3 Refining the Present Design At the present time there are sufficient data to describe the concentrations and distribution of trace chemicals in San Francisco Bay. Part of the RMP's planning for the next five-year period should be an evaluation of how well the present design meets the original objectives. This should be done with an eye toward assessing the numbers of sites, the frequency of sampling, and/or the numbers of constituents analyzed. Such evaluations might have the effect of reducing sampling and analysis efforts while preserving the core RMP program and meeting the original objectives (see Chapter 2, Box 1). That might well enable the RMP to free up financial resources for studies related to broader RMP objectives (see Chapter 2). The question of sampling site location has already been touched on (see above); other examples of issues related to modifications of the study design follow: The RMP has begun to develop regressions between total aqueous concentrations of many trace contaminants and total suspended solids (TSS). This should be expanded to test the validity of using only TSS measurements to monitor exceedances of water quality criteria. It seems that this should be possible because invariably those exceedances are due to high concentrations of particle-bound copper, mercury, nickel, or PCB. These data strongly suggest that present exceedances are due in large part to the historical pool of contaminants in Bay sediments. The Review Panel suggests that this implication be considered in any attempt by the RMP to link water quality patterns to current sources of contamination. The number of sites should also be examined. As presently designed, the RMP aims to describe trace contamination in the open part of San Francisco Bay along its spine. With the data now in hand, the possibility should be tested that fewer stations would be sufficient to define the chemical condition of each major region of the Bay. The original objectives assumed that trace chemical concentrations and toxicity varied with season or place in the water year. Now, with four years of seasonal data, that assumption can be tested, although not very powerfully. If it were determined that the influence of season is weak, then less frequent sampling might be considered. This would be particularly true if such seasonality were shown to have little impact on regulatory decisions, source detection, or longer-term trends. There are data for only four years, which are too few to reliably estimate interannual trends. However, with data from the USGS on chemical concentrations in dated sediment cores, it should be possible for the RMP to estimate the time required for particles to be buried so deeply that they no longer contribute trace contaminants to surface sediments or (through resuspension) to TSS. This needs to be known in order to gauge the number of years required for surface sediment and TSS to respond to changes in contaminant inputs. The preceding discussions are intended only as examples. Other parts of the RMP program should also be reevaluated. Such reevaluations of initial assumptions are an extremely useful part of any periodic program review and the Review Panel strongly recommends that they be included as a key part of the development of the program's next five-year plan. 3.4 Setting Study Goals As emphasized throughout this chapter, a prerequisite for effective study design is a description of explicit study goals and objectives. The Review Panel recognizes that the RMP's original objectives have served it well by providing the direction needed to begin essential high-quality baseline and trends monitoring. However, the RMP must develop more precise statements of objectives if it is to fulfill its potential, make effective use of the data already gathered, and meet the information needs of its Participants. As described in the proposed new Framework for the RMP (SFEI 1997a), several levels of detail are needed. At the first, aims may be relatively imprecise, e.g., to "track the health of the Bay," "monitor trends," and so on. However, these must eventually be defined and made more quantitative in order to adequately focus monitoring and special studies. In addition, target levels for uncertainty must be set to put some boundary around sampling and analysis effort. A necessary element in creating these more detailed goals and thereby focusing the monitoring design is a set of expectations, or predictions, about how the Bay will respond in the future. For example, the expectation that measures to control contaminant inputs to the Bay will lead to measurable reductions of contaminants in Bay sediments in five years would dictate one kind of trend monitoring program. The expectation that measurable reductions will only be evident in 50 years (because of existing loads and their residence times) would lead to a very different trend monitoring program. The Review Panel believes that the RMP would benefit immensely from even a relatively preliminary effort to develop such predictions, based on system-wide mass balances, residence times of key pollutants, and other modeling parameters. Setting study goals and objectives is not a strictly technical
exercise. This is because answers to the kinds of design questions
posed throughout this chapter depend on knowing what managers and
decision makers want to know. This will involve 3.5 Recommendations
|
| 4.0 Quality Assurance and Data/Information Management and Transfer Summary: Quality assurance and quality control (QA/QC) are integral to the success of the RMP. The RMP is clearly producing high-quality data and reporting accurately on them. There are no critical shortcomings in the RMP database or in the QA/QC procedures that require immediate attention. There is a commendable emphasis on numerical data quality objectives, quality assurance and control, use of state-of-the-art laboratory analysis methods, careful tracking of sample custody and data, and well-written reports. The scope, content, and format of data and information products should reflect the goals of the RMP and the intended uses of its information in decision making and management. What problems exist with RMP data and information products are related to the fact that overall RMP goals are not well-defined and fail to guide choices about data management approaches, data presentation methods, or data access standards (see also Chapter 3).
4.1 Quality Assurance/Quality Control of Laboratory Analysis RMP contract labs have been routinely participating in the annual NOAA-sponsored intercalibrations, in which many labs measure the same unknowns. In 1995, the BADA labs also participated. Intercomparison exercises such as this deter "analytical boredom" (i.e., routine analysis leading to carelessness). Without intercomparisons, the only checks of accuracy are either standard reference materials (SRMs), in which the "correct" result is known, or previous years' results with samples from the same location. Data from previous years' samples are not a check on accuracy, but they can serve to flag absurd results. Some specific QA/QC questions are considered in the following paragraphs. Polychlorinated Biphenyls (PCBs): The individual PCB congeners measured in the RMP are not all common to all samples. For example, among water samples there are small differences in the congener list between 1993 and 1994 and then a larger difference in 1995. This reflects the changes in laboratories in 1995. In 1995, 45 PCB congeners were measured in water, bivalves, and sediment. For 13 of the 45 congeners, coelution is a problem that is acknowledged by the lab doing the analysis of bivalves and sediment. In contrast, the laboratory analyzing water samples does not acknowledge coelution of PCB congeners as a problem. In 1995 there are some PCB congeners that were measured only in water and 28 that were not measured in water but were measured in bivalves and sediment. One PCB congener was measured only in bivalve tissues. Adding up all the congeners to get "Total PCBs" is not appropriate when the totals for each type of sample do not include the same congeners. This is not a major issue, since the uncommon congeners usually do not add much to the total. However, the rationale for such inconsistencies should be documented or they should be removed. Polycyclic Aromatic Hydrocarbons (PAHs): There may be a large discrepancy in aggregated total PAH concentrations between sample types. In 1995, 25 individual PAH compounds were measured in bivalves and in sediment, but only 13 of those were also measured in water. The appendices in the Annual Reports describing methods for trace organic analyses are incorrect with regard to bivalves and sediments, since the methodological descriptions apply only to the analysis of water and suspended particles. The derivation of the MDLs (method detection limits) listed throughout the RMP reports, and in the database, is not given. Standards and Controls: The measures of accuracy and precision listed in RMP reports for analyses measured in water are said, in footnotes, to be from periodic analysis of standard samples within the analytical stream. There are no such footnotes for the sediment and bivalve analyses, so it is not clear that controls were run. Since there are accuracy and precision measurements, it is obvious that sediment and tissue standard reference materials were analyzed, but it is not clear that they were run in every sample batch. Method Detection Limits (MDLs): MDLs have not been clearly documented. Since MDLs may be determined in several different ways it is essential that the MDLs used in the RMP analyses be carefully described for each analytical protocol, and each analytical laboratory. Censored Data: If measured concentrations are less than MDLs they are edited to "<" or to "ND" in the RMP database. These designations are not equivalent. Real numbers have been censored to "<" in the data. Users of the data can consider measured concentrations less than MDLs to be zeros, set them to the MDLs, use a random number in between, or apply another convention. 4.2 Design of the RMP Data System to Meet Programmatic Objectives The design of the RMP data system is consistent with the program objectives of compiling baseline data, quantifying seasonal and annual trends, and analyzing compliance of water and sediment quality with criteria. There is little evidence in the RMP reports that historical data from other monitoring programs in the San Francisco Bay area have been integrated with the RMP. This integration must be carried out in order to attain the objective of determining long-term trends, since analysis of long-term requires implies integration with historical data sets and data from other Bay Area monitoring programs. The data from special and pilot studies do not appear to be treated the same as the data from the core program. They are typically not included in appendices to the annual reports nor available via SFEI's World-Wide-Web site. Because of this they are less accessible, making it more difficult to quality-assure and interpret the entire RMP dataset and to achieve maximum benefits from special and pilot studies. 4.3 Data Management, Quality Assurance/Quality Control, and Documentation The staff of SFEI, AMS, and subcontractors are highly skilled in data management. Given the changes that are underway in handling RMP's data (i.e., the direct flow of data from contract laboratories to SFEI), the role of SFEI's data management staff will become even more critical. Fortunately, SFEI's data management staff (in particular Jung Yoon, database manager, and Todd Featherston, network administrator) are trained and skilled in such areas as systems and database design. While the hardware and software aspects of data management are being handled very thoroughly and professionally, the Review Panel believes there is insufficient documentation describing the data path, i.e., exactly what is done with the data between the moment of sampling and the ultimate release of data to users. For example, there is no description of what QA checks are performed by AMS, subcontractors, and SFEI. Based on its examination, the Review Panel has concluded that the data are being managed better than is documented. Interviews and inspections showed that spreadsheet files are spot-checked against data entry sheets, macro-generated flat files are spot-checked against the source spreadsheet files, station names are changed for consistency, and checks against data quality objectives and for outlying values are made. However, these QA/QC measures and other data-handling procedures should be documented to ensure consistency, provide visible operating standards, and enhance the credibility of the data. The lack of such documentation does not invalidate any specific values in the RMP database; rather, proper documentation would protect the investment that has been made in the RMP and give more credibility to the RMP database. More thorough documentation would also make the data more useful to other scientists and buttress the quality of the data in case of scientific or legal challenge. The RMP data are very accessible, either directly from the RMP Oracle database (via query from the SFEI World-Wide-Web site) or by request from users to SFEI staff. Data are considered publicly available when the corresponding RMP annual report has been published. Before that time, requests are considered on an ad hoc basis by SFEI. A data distribution policy is in place and precedes access to RMP raw data on the SFEI Web-site. The data storage system is very appropriate for the needs of the RMP. The RMP database is backed up on a daily basis to another disk drive/directory on the network, and weekly backup tapes are stored on-site in a fireproof safe (a 3-month history can be restored). Backup tapes are not removed to an off-site location. Using only on-site storage could result in a catastrophic loss of the data should an accident occur at the Richmond Field Station. The Review Panel believes that off-site storage of RMP data is critical, logical, and easy to implement. 4.4 Reporting Mechanisms and Data Distribution SFEI regularly produces several data/information-related products specified in the memorandum of understanding (MOU) between the Regional Board and SFEI. These include: an annual report with analysis and interpretation (printed), data available for public review (available through SFEI's World-Wide-Web site and, on request, in hard copy), and an annual meeting of the program sponsors. In addition, SFEI produces the newsletter Regional Monitoring News and stand-alone pieces such as Monitoring Results (prepared as an insert to the IEP newsletter). The contents and format of the annual report, the database, and the annual meeting are not specified in the MOU, nor is there any specification of the level of interpretation to be applied to RMP data. The Review Panel believes that more explicit guidance is needed for RMP data analysis, interpretation and presentation. Such guidance can only come from RMP Participants, and only after RMP Participants have fully articulated their needs and expectations (see Chapters 3 and 5).
4.5 Usage of Data/Information Products It is not clear whether RMP information products meet the needs of its users in all cases. The Review Panel found that RMP products are not used to a uniform extent by the Regional Board and RMP Participants, nor is there a full appreciation for the extent to which RMP results are used. Users' needs are diverse and include making pollutant loadings estimates, developing sediment criteria, assessing general compliance with water quality objectives and criteria, reducing testing requirements, assessing the relative likelihood of impacts from different contaminants, determining sources, and providing public relations benefits. Mismatches between products and needs arise primarily from the lack of a clear understanding among SFEI, the Regional Board, and the Steering and Technical Review Committees concerning the amount and types of information needed and how these should be packaged into particular products (see also Chapters 3 and 5). 4.6 Recommendations
|
| |
5.0 Organizational Relationships and Capacities Summary: The authorities and relationships that structured the RMP were defined in the original documents that established the program. These provided sufficient guidance for the RMP's first five years, during which the program focused on implementing the initial five-year plan. During this period, the parties to the RMP have also established a cooperative, consensus-based management process. This has served to foster participation and commitment among a diverse group of interests. At the present time, however, this management system does not provide the structure and definition needed to guide the program through its next phase of development. The Review Panel strongly recommends that the RMP give serious attention to better defining the roles, responsibilities, and authorities of its members; develop more detailed and formal decision-making procedures; and increase its ability to address and resolve healthy conflict about the program's overall direction. The Review Panel believes that the development of a new five-year plan for the RMP provides an ideal opportunity to address these issues.
5.1 Introduction The formal, legal relationships among the organizations involved in conducting the Regional Monitoring Plan (RMP) are described in Resolution No 92-043 of the California Regional Water Quality Control Board, San Francisco Bay Region (the Regional Board), and the MOUs between the Regional Board and the San Francisco Estuary Institute (SFEI; then named the Aquatic Habitat Institute [AHI]) dated 1992 and 1996. Resolution 92-043 stated the processes followed, and the authorities used, by the Regional Board in establishing the RMP as part of the Regional Monitoring Strategy under the umbrella of the San Francisco Estuary Project (SFEP). The original implementation plan for the RMP was charged to the Executive Officer of the Regional Board: "...the...Executive Officer shall work with the selected dischargers to prepare an implementation plan for the RMP including a schedule for submittal..." In a memorandum of understanding (MOU) the Regional Board and AHI (SFEI) agreed to establish a Steering Committee for the RMP comprising at least one member from each discharger category. The Steering Committee was charged with the responsibility to work on "... such issues as allocation of future program costs, selection criteria for contractors, participation in proposal review and selection, review of progress, technical review of program results, and evaluation of effectiveness of the program on a periodic basis." Resolution 92-043 and the text of the MOU make it clear that the Regional Board, SFEI, and the Steering Committee (representing the Participants) have authority to prepare the Study Plan for the core program. The study plan was described in the MOU as "... a detailed plan for implementing the baseline portion of the Program." The Technical Review Committee (TRC) was established by action of the Regional Board, SFEI, and the Steering Committee as comprising a representative from SFEI, a representative from the Regional Board, and at least one member from each discharger category. The TRC was assigned the responsibility of assisting SFEI in "... developing annual work plans and five-year plans that are technically sound and consistent with the policy guidance of the Steering Committee and the Regional Board." These original definitions of roles, responsibilities, and authorities served to initiate the RMP and provide broad guidance during its early years. However, this original authorizing language left the development of the details of day-to-day planning and decision making to the program's participants. While the processes that arose during the first five years were sufficient to accomplish the RMP's initial set of objectives, the Review Panel believes they do not provide an adequate basis to meet the challenges that currently confront the RMP. The following sections review the decision-making dynamics within the RMP and recommend ways these could be enhanced to help implement the recommendations listed in the other chapters in this report. 5.2 Consensus-Based Decision Making Within the RMP In a formal sense, each organization participating in the RMP has a specified supporting role, defined in general terms, in the original Regional Board Resolution and Memoranda of Understanding (MOUs) and carried out under the Regional Board's overriding regulatory responsibility. While these founding documents outlined basic relationships and responsibilities, they left the day-to-day management approach of the RMP to be devised by the parties themselves. The parties to the RMP envision themselves as a coalition of organizations with a cooperative working relationship among "equals" with roughly equivalent influence and organizational authority. The Review Panel considers this a classical consensus management model and found that the program's participants in general concurred with this judgment. Thus, the RMP developed from 1993 to 1997 without requiring detailed written policies and procedures that identify which parties have specific responsibility for recommending, reviewing, and approving programmatic content. Nor has there been an explicit process for resolving conflicts. In fact, the actual working relationships within the RMP are still developing and are not consistently perceived by representatives of different groups, a not unusual situation in a complex consensus-based program only five years old. The existing consensus management approach was appropriate early in the program because it allowed for widespread involvement in the development of the RMP and helped build commitment among the parties to the program. In addition, consensus management is most efficient when the dispersed power of the coalition is focused and exercised by a single spokesperson or small inner circle (Thompson, 1967). At the inception of the RMP, this role was played by Mr. Michael Carlin, whose vision for the RMP helped to bring it into being. Mr. Ritchie's (the Regional Board Executive Officer at the time) authority and Mr. Carlin's stature within the Regional Board staff made possible the effective management of the coalition as the scope and funding of the RMP evolved to its present level. As such a program matures, however, consensus decision making may in fact limit the ability of the organiza |